Archive

Archive for the ‘IPV6’ Category

Ipv6 Logo Interoperability Test using Tahi Vel tool

August 23, 2013 11 comments

I found that lot of queries in the tahi forums are regarding setting up of the Ipv6 interoperabilty testbed and thought of sharing my experience which may help others.

This is a continuation of my previous post “IPv6 logo certification using Tahi Self test tool

As mentioned there to get the ipv6 logo certification, two tests are mandatory,

  1. ipv6 logo conformance test using the Selftest tool
  2. IPv6 interoperability test using the Vel tool

Goal of this test

The primary goal for this test is to make sure that, the unit under test’s (UUT) ipv6 implementation is interoperable with other vendor implementation.

Hardware Requirements for the test.

  • VEL tool.
  • 2 x PC’s(Reference Node) running free BSD OS with 3 NICS
  • 1 x PC(Vel Manager Node) running Free BSD OS with 1 NIC
  • 1 x PC (Dumper Node) running Free BSD with 4 NICs
  • 4 X PC’s (Target Nodes) running 4 different OS with 4 NICs each. Recommended Target OS are         – FreeBSD 7.0
            – NetBSD 4.0.1
            – OpenBSD 4.3
            – Fedora9
    – KAME
    – USAGI

*  I have used the bold ones as my target OS

So as we can see we need total 8 PC’s/Servers in total. We can reduce the number of PCS to 5 by running all the 4 Target OS installed in 1 single PC as well. Also please note the number of NICs which are required as well. Because of this requirements its a very difficult to have this topology. but fortunately the new virtualization technologies can help us with solving the issue and this post is about how we can optimize/achieve the same topology using a virtualization enabled server using  Vmware’s Esxi.

Optimized Setup/resources which I used

  1. Server Hardware: Cisco UCS C250M1 Rack mount server running ESXi 5.1 (Instead of this any server/hardware which is supported by ESXi can be used as well)
  2. Number of Physical Nics on the server: 1
  3. Virtualization Software/Hypervisor: ESXI 5.1

Understanding the topology for Interoperability test

First of all lets understand how to read the topology which is mentioned in the Vel tool’s read me file. Following is the topology mentioned in the README file of vel tool

                                      Network 1
        ------+--------------+-----------+------+-
              |              |           |      |
              | 1            |           | 1    +-------------+
       TG +-----------+      |        +------+                |
   +------| REF-1 (V) |      |        | LOGO |                |
   |      +-----------+      |        +------+                |
   |          | 2            |                                |
   |          |              |        Network 2               |
   |    ------+------+------------+------+-----+--            |
   |                 |       |    |            |              |
   |               1 |       |    |            +------+       |
   |       TG +----------+   |    |                   |       |
   | +--------| REF-2 (V)|   |    |                   |       |
   | |        +----------+   |    |                   |       |
   | |             2 |       |    |                   |       |
   | |               |       |    |   Network 3       |       |
   | |  ------+------+------------------------+---    |       |
   | |        |              |    |           |       |       |
   | |        | 3            |    |           |       |       |
   | |  TG +----------+ 1    |    |           |       |       |
   | | +---| TAR* (V) |------+    |           |       |       |
   | | |   +----------+           |           |       |       |
   | | |      | 2                 |           |       |       |
   | | |      +-------------------+           |       |       |
   | | |                                      |       |       |
   | | |                                      |       | 2     |
   | | |     Network for TG                   |  +---------+  |
  -+-+-+----+-------+-                        +--| dump (V)|--+
            |       |                          3 +---------+ 1
         TG |       |                                 | TG
      +---------+   +---------------------------------+
      | vel mgr |
      +---------+

the above one doesnt give a clear picture on how the connections are. So lets see how the topology can be interpreted.

  • Each Square/rectangular boxes are devices or PCS
  • LOGO : device under testing or your device.
  • vel mgr : vel manager, device on which velm program runs (Runs Free BSD OS)
  • TAR* : 4 interoperable devices from other vendors devices (Target nodes running target OS) including 2 hosts and 2 routers.
  • REF-1 : reference node running Free BSD OS
  • REF-2 : reference node running Free BSD OS
  • dump : packet monitor running Free BSD OS
  • (V) : device which vela and velo programs run and they are TAR*, REF-1, REF-2 and dump
  • The number around the box/device indicates the Network interfaces
  • The dashed lines (—) are the links/connections from the NIC cards
  • Network1, Network2 and Network3 are Hubs (Remember its a hub not a switch)
  • Network for TG is the management network and can be a switch.
  • “+” sign on the dashed line indicates that those two lines merge there, if there is no “+” sign that means those lines dont merge.

How to setup the same topology in the virtualized Server.

The same setup can be easily set up using Vmware’s ESXi.

  1. VM Creation: Create VMs corresponding to all the 8 devices in the topology in the same ESXi server. While creating VMs, allocate the required number of Virtual NICs as well to the VM.
  2. Vswitch Creation:  Now our job is to simulate three different network hub’s(network1, network2 and Netowrk3). For this in Vsphere Client go to ESXI Host->Configuration->Networking->Add networking–>Virtual machine(as the connection type)->Select no Physical adapters->Give a network label(network1)->Finish. Repeat the same for Network 2 and network 3. For Network TG select the physical NIC corresponding to the NIC which is conencting to the Management NIC.
  3. Mapping Network Labels to Virtual NICs: Now map the virtual NICs of each of the VM to the appropriate Network labels(Right click on the VM–>Edit Settings->Select the NIC adapter->Enter the network Label as Network1/2/3 depending on the topology.
  4. Configuring the Vswitches to behave like a hub: Another requirement is to configure the vswitches created in step 2 to behave like a hub. For this go to ESXI Server->Configuration->Networking->Select the Vswitch Name(Networ1/2/3)->Properties->Security Tab->Enable promiscuous Mode and Change it to Accept and OK.
  5. IP address Assignment:  The Virtual NICS which are mapped to Network TG will be assigned with IP address which is used for management as well as for velmgr to interact with vel clients.

Installing and Running the tool.

This is already covered in the README file of the Vel tool.

Important input file for the vel tool is a file called “config” in the “vel-5.0.1/interop_ph2” directory.

Following are the most important items which needs to be changed.

# LOGO TYPE: type of your device [host or router]
LOGO_TYPE router
# LOGO_NAME: name of your device
LOGO_NAME CISCO Nexus 7000

LOGO_IF1 Ethernet1/1
LOGO_IF1_MAC 00:00:00:00:00:01
# LOGO_IF2 : interface name and mac address on your device
# If LOGO_TYPE is router, you need to edit LOGO_IF2
LOGO_IF2 Ethernet1/2
LOGO_IF2_MAC 00:00:00:00:00:02
Similarly put the corresponding NIC mac address for the other devices in the file. Mac address can be found from the ifconfig command output.

Edit the following to change the ip address for the management or TG network
TAR1_IFTG eth0 "Put the interface name here"
TAR1_IFTG_V4_ADDR 192.168.20.10
TAR2_IFTG eth0 "Put the interface name here"
TAR2_IFTG_V4_ADDR 192.168.20.11
TAR3_IFTG eth0 "Put the interface name here"
TAR3_IFTG_V4_ADDR 192.168.20.12
TAR4_IFTG eth0 "Put the interface name here"
TAR4_IFTG_V4_ADDR 192.168.20.13
REF1_IFTG eth0
REF1_IFTG_V4_ADDR 192.168.20.10
REF2_IFTG eth0
REF2_IFTG_V4_ADDR 192.168.20.101
DUMPER_IFTG eth0
DUMPER_IFTG_V4_ADDR 192.168.20.

Some Tips

  • Make sure the firewall is disabled in all the devices which you are using. Else, vel manager may fail to connect to the vel clients.
  • Some of the tools wont be there in fedora by default. rtadvd is such a tool which I had to install.
Advertisements
Categories: IPV6, Technical

Ipv6 Logo Certification Using TAHI Self Test tool

March 16, 2012 97 comments

The intent to write this blog is to help anyone who would like to use the free TAHI Selftest Tool
for the IPv6 Logo Certification. Before I write anything further, I should say that whatever I write
here is based on the support which I got from the TAHI forums and with out those people it wouldn’t have been possible.

Special thanks to Joseph Mattress,Sean Cavanaugh, Mark Atkinson, Arul Murugan, www.ipv6style.jp
and many other Tahi forum members

Note: Most of the things which I write here are applicable only if you want to run TAHI using the
Vremote scripts.
Also most of the issues I discusss here are based on my testing with Cisco
Nexus 7000 dataceneter Switch.So some of the things may not be applicable for other
vendors/devices.

Background on IPv6 Logo Certification and TAHI Self Test Tool

The IPv6 Forum, a world-wide consortium, with a key focus to provide technical guidance for the
deployment of IPv6, launched a single world-wide IPv6 Ready Logo Program (conformance and
interoperability testing).The IPv6 Ready Logo Program is a conformance and interoperability testing
program intended to increase user confidence by demonstrating that IPv6 is available now and ready
to be used.

The IPv6 Forum has created the IPv6 Ready Logo Committee (v6LC), to manage the IPv6 Ready Logo Program. It comprises representatives from equipment vendors, service providers, academic institutions, IPv6 organizations, members from the TAHI Project (Japan), the University of New Hampshire InterOperability (UNH-IOL)(USA), IRISA/INRIA (France), European Telecommunications standardization Institute ETSI (Europe), Telecommunication Technology Association TTA (Korea), Beijing Internet Institute BII (China), ChungHwa Telecom Labs CHT-TL (Taiwan) and Japan Approvals Institute for Telecommunications Equipment JATE (Japan).

IPv6 Ready Logo Phase Series – Phase-1, Phase-2 and Phase-3

The IPv6 Ready Logo series of tests were progressively enriched, from a minimum coverage with Phase-1 to a more complete coverage with the Phase-2 and later on with Phase-3.

  • Phase 1 (Silver) Logo: In a first stage, the Logo will indicate that the prduct includes IPv6 mandatory core protocols and can interoperate with other IPv6 implementations.
  • Phase 2 (Gold) Logo: The “IPv6 ready” step implies a proper care, technical consensus and clear technical references. The IPv6 Ready Logo will indicate that a product has successfully satisfied strong requirements stated by the IPv6 Logo Committee (v6LC).
  • Phase 3 Logo: Currently being planned, will be the same as the Phase 2 Logo in terms of requirements except that the extended test category for IPsec will be mandatory.

More details on IPv6 Ready Logo, Please visit http://www.ipv6ready.org/

TAHI Selftest Tool is a free IPv6 Conformance testing Tool which has a predefined tests as stipulated by the ipv6ready.org.

More details on TAHI, Please visit http://www.tahi.org/ume/

Must Refer Links before you proceed further [Explains how to get the certification and Selftest tool in Details]

Courtesy: www.ipv6style.jp

About IPv6 logo Certification

Getting IPv6 Logo Certification

Interoperability Test Links

Hardware Requirements for TAHI

  1. A stand alone PC/Virtual machine which has 3 NIC cards (1 for management connectivity, and 2 other NICS for connecting to the Unit Under Test which will be referred from hereon as UUT.
    Note: I had tried TAHI on a stand alone PC as well as a VM and following were my configurations
    Standalone PC:  CPU: Intel(R) Xeon(TM) CPU 3.00GHz (3000.12-MHz 686-class CPU), Logical CPUs per core: 2
    Memory : 1 GB
    VM: CPU: Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz (2925.69-MHz 686-class CPU)
    Memory: 256 Mb RAM

FreeBSD9.0 and TAHI Notes

If you are trying to install TAHI v6eval3.3.2 which is the latest version (as per March2012), there are few things which eneds to be taked care. Please refer shaun’s blog (Sean’s Blog on V6eval installation issues in Free BSD 9.0)regarding the same and the comments on that. Thanks a lot Sean for this tip.
Following is the wayto do it, incase the original site gets changed.
Create a file called patch-utmpx with the following contents in the extracted v6eval3.3.2 folder. I have it extracted in /root/

FreeBSD-TAHI# cat /root/v6eval-3.3.2/patch-utmpx
— lib/Cm/CmMain.cc
+++ lib/Cm/CmMain.cc
@@ -47,7 +47,7 @@
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
-#include <utmp.h>
+#include <utmpx.h>
#include <time.h>
#include <pwd.h>
#include <sys/time.h>
@@ -128,19 +128,28 @@

 //———————————————————————-
// &frac34;ã³&sup2;&sup2;òÀÏ&frac34;ðÊóºîÀ®
-static struct utmp *myUtmpEnt(FILE *in,struct utmp *u) {
–       int s=ttyslot();
–       if(s<0||fseek(in,sizeof(struct utmp)*s,0)<0||
–               fread(u,sizeof(struct utmp),1,in)==0) {return 0;}
–       return u;}
void CmMain::makeCatch2Eye(STR p) {
static char catch2[]=” on %*.*s:%-*.*s from %*.*s”;
–       struct utmp ux[1], *u; FILE *in;
–       if((in=fopen(“/etc/utmp”,”r”))==NULL) {return;}
–       u=myUtmpEnt(in,ux); fclose(in);
–       if(!u) {return;}
+       struct utmpx ul, *u;
+       const char *tty;
+
+       tty = ttyname(0);
+       if (tty == NULL)
+               tty = ttyname(1);
+       if (tty == NULL)
+               tty = ttyname(2);
+       if (tty == NULL)
+               return;
+       if (strncmp(tty, “/dev/”, 5) == 0)
+               tty += 5;
+       strncpy(ul.ut_line, tty, sizeof(ul.ut_line));
+       setutxent();
+       u = getutxline(&ul);
+       endutxent();
+       if (u == NULL || u->ut_type != USER_PROCESS)
+               return;
#define A(a)sizeof(a),sizeof(a),a
–       sprintf(p,catch2,A(u->ut_line),A(u->ut_name),A(u->ut_host));
+       sprintf(p,catch2,A(u->ut_line),A(u->ut_user),A(u->ut_host));
#undef A
return;}
void CmMain::makeCatchEye(const STR pgmName) {
FreeBSD-TAHI#

Now patch the lib/Cm/CmMain.cc file using the following command
FreeBSD-TAHI# pwd
/root/v6eval-3.3.2
FreeBSD-TAHI#
FreeBSD-TAHI#patch lib/Cm/CmMain.cc </usr/include/patch-utmpx

Following are some useful FreeBSD commands which will come handy

  1. Uninstalling a application which is installed via ports
    In FreeBSD to uninstall an application which is installed via ports go to the ports directory of that application and issue the command “make deinstall”
    eg:
    FreeBSD-TAHI# cd /usr/ports/lang/p5-Expect
    FreeBSD-TAHI# make deinstall
    ===>  Deinstalling for lang/p5-Expect
    ===>   Deinstalling p5-Expect-1.21
    FreeBSD-TAHI#
  2. Patching a file
    Patching is used mainly to update a source code to make changes easily. The most commonly used syntax for the same is
    patch targetfile <patchfile
  3. Checking the perl version: perl -v
  4. Checking the perl modules present in the system: Type the commands “instmodsh” and then press “l”

Step by step guide for Installing TAHI Selftest Tool

  1. Download FreeBSD 7.2 or higher from the internet. I choose Free BSD 7.3 over higher versions as the above versions had a bug with Intel NICS, where the NIC card was not getting detected. (This issue is solved in FreeBSD 9.0, But please see the FreeBSD 9.0 Notes before proceeding)
  2. Install Free BSD on to a stand alone machine/Virtual machine
  3. Make sure you have got internet connectivity from the FREE BSD machine. If you are using a proxy server, following commands will set configure proxy. Replace the address with your address
    setenv http_proxy http://proxy.com:8080
    setenv ftp_proxy http://proxy.com:8080
    setenv HTTP_PROXY=http://proxy.com:8080
    setenv FTP_PROXY http://proxy.com:8080
    Note: For FreeBSD ports(similar to apt-get or yum) to work with proxy you may have to change the following as well
    So please edit the following file/create if its not there
    /etc/make.conf Add the following entries
    FETCH_ENV=  HTTP_PROXY=http://proxy.com:8080
    FETCH_ENV+= FTP_PROXY=http://proxy.com:8080
  4. Download the latest v6eval tool from http://www.tahi.org/release/v6eval/v6eval-3.3.2.tar.gz
  5. Download the latest SelfTest Tool from http://www.tahi.org/logo/release/Self_Test_5-0-0.tgz
  6. Download the v6eval-remotes scripts from http://www.tahi.org/release/v6eval/v6eval-remotes-3.0.tar.gz
  7. Before installing the above tools we need to install the below (This is a copy paste from INSTALL.dummies and INSTALL.v6eval which can be found in v6eval package)
    Install perl5
    Install perl mmodules
    Install IO-Tty by cd /usr/ports/devel/p5-IO-Tty && make install
    Install IO-Pty by cd /usr/ports/devel/p5-IO-Pty-Easy && make install
    Install Expect by cd /usr/ports/lang/p5-Expect && make install
    Install Digest-MD5 by cd /usr/ports/security/p5-Digest-MD5 && make install
    Install YAML by cd /usr/ports/textproc/p5-YAML && make install
    Checking whether the perl modules got installed
    FreeBSD-TAHI# instmodsh
    Available commands are:
    l            – List all installed modules
    m <module>   – Select a module
    q            – Quit the program
    cmd? l
    Installed modules are:
    Digest::MD5
    Expect
    IO::Pty::Easy
    IO::Tty
    Locale::gettext
    Perl
    V6evalCommon
    V6evalRemote
    V6evalTool
    YAML
    cmd?

    If you have all the above modules installed then you are good to go.
    Install apache Webserver(Recommended as the results are in html format), links a text browser and vim if required
    Start Apache webserver by issuing “/usr/local/sbin/apachectl start”
  8. Install V6eval by doing the following commands
    Extracting the package
    % cd $SOMEWHERE
    % tar zxvf v6eval-X.Y.Z.tar.gz
    Compiling & installing the tool
    % cd $SOMEWHERE/v6eval-X.Y.Z
    % make
    # make install
  9. Make bpf special device.
                    # vi /etc/rc.conf
    # vi /etc/devfs.rules
                    /etc/rc.conf
    —————————————————————-
    devfs_system_ruleset=”devfsrules_unhide_bpf”
    —————————————————————-
                    /etc/devfs.rules
    —————————————————————-
    [devfsrules_unhide_bpf=1025]
    add path ‘bpf*’ user root group wheel mode 0660 unhide
    —————————————————————-
            The user who executes conformance test need to belong in wheel group.
                    # vi /etc/group
  10. Configure serial line
                    # touch /var/log/aculog
    # chown uucp:dialer /var/log/aculog
    # chmod 660 /var/log/aculog
            The user who executes conformance test need to belong in dialer group.
                    # vi /etc/group
  11. Interface configuration
            disable IPv6 support on network interface used in testing.
                    # vi /etc/rc.conf
                    /etc/rc.conf
    —————————————————————-
    ipv6_enable=”NO”
    —————————————————————-

    At least, one or two network interfaces used in testing
    must be ‘up’ status.
                    # vi /etc/rc.conf
                    /etc/rc.conf
    —————————————————————-
    ifconfig_<Link0_device_name>=”up”
    —————————————————————-
    If NUT is a router, you need to specify also Link1.
  12. Install the V6eval-remotes package (In my case I have taken the cisc-ios scrips as a base-line and changed the CLIS to match with NXOS)
    Note: Installation is not a must. Copying the script directory to /usr/local/v6eval/bin/ and giving all the *.rmt scripts execution permission is more than enough
    Find an OS similar to the sample OSes which are given along with V6eval-remotes package. IN my case I took cisco-ios as baseline.
  13. Edit the V6evalRemote.pm file and make the following changes (Most of these are based on my OS)
    Search for cisco-ios and replace it with nxos-svi (Not mandatory though it would be good)
    Replace the cisco-ios propmpts and commands with your Operating Systems prompts and commands wherever applicable (eg: login: , password, # etc)
    Edit the *.rmt scripts inside the cisco-ios directory with your OS commands. Some tweaks in timing may be required based on the UUT OS.
    Rename the cisco-ios directory to nxos-svi for consistency across everywhere
    Run a sample script to make sure script is able to connect to UUT (please change the parameters according to your config”
    perl /usr/local/v6eval/bin/nxos-svi/reboot_async.rmt -t nxos-svi -u jerin -p jerin -d 10.104.104.117 -o 1 timeout-120
    where -t is the device to be picked from V6evalRemote.pm
    -u username to login to the uut
    -p password to login to the uut
    -d Device which is sepcified in the tn.def (cuad0 if you are using serial port to connect, in my case mgmt ip of the device as I use telnet)
    If the script is able to login then we are done.
  14. Configuring the nut.def (I have mentioned only the relevant areas which needs to be changed. uut related configs are done in this file)
    Note: Please be careful while editing this file as small mistakes like additional spaces/TABS in this file may cause TAHI not to run)
    change the following

    nut.def.sample

    nut.def

    HostName        target.tahi.org HostName        <your_device_prompt>
    Type            router set the type to host/router/special device according to your need
    User            root User            <username_of_ur_device>
    Password        v6eval Password        <password_of_ur_device>
    System          manual System          <ur_required_script_folder_name>
    eg: System          cisco-ios
    Link0           fxp0            00:00:92:a7:6d:f5 Link0           <uut_int_name_1>            <ur_routermac>
    eg: Link0           Vlan10          00:26:98:1b:e1:41
    Link1           fxp1            00:00:92:a7:6d:f6 Link1 <uut_int_name_2> <ur_routermac>
    eg: Link0           Vlan20          00:26:98:1b:e1:41


    Note: TAHI recommends configuring the interface as a Layer-3 port. But since our Cisco N7k is a Multi layer switch and
    since I want to run TAHI across multiple line cards (some line cards have only layer-2 support), I used SVI’s (Vlan interfaces)
    TIP: Best way to get this change done is via sed command
    cat /usrlocal/v6eval/etc/nut.def.sample | sed -e ‘s/find1/replace1/find2/replace2’

  15. Configure the tn.def(test tool related configurations are done in this file) according to your requirements

    tn.def.sample

    tn.def

    RemoteDevice    cuad0 RemoteDevice    <mgmt_ip/serialport>
    eg: RemoteDevice    10.104.104.117
    RemoteDevice    cuad0
    Link0           de0             00:00:00:00:01:00 Link0           <freebsd_int_name1>             00:00:00:00:01:00
    eg: Link0           em1             00:00:00:00:01:00
    Link1           de1             00:00:00:00:01:01 Link1           <freebsd_int_name2>             00:00:00:00:01:01
    eg: Link1           em2             00:00:00:00:01:01


    Note: em1 in the example tn.def is connected to the Vlan10 in nut.def

  16. If you have installed webserver, then copy the Self_Test_5-0-0.tgz to /usr/local/www/data/ and then untar it
  17. Now cd to Self_Test_5-0-0/ and to execute phase 1 issue “make ipv6ready_p1_router” and to execute phase 2 “make ipv6ready_p2_router”

UUT Configuration before running the script

For the default TAHI tests to work, the following EUI-64 IPv6 prefix addresses needs to be configured on the device

Interface connected to TAHI Link0: 3ffe:501:ffff:100::/64

Interface Connected to TAHI Link1:  3ffe:501:ffff:101::/64

My Topology

Topology

Interface Vlans/SVis Vlan10 and Vlan 20 are configured on UUT.
N7k1 acts a L2 switch which relays the packet from freebsd/TAHI to UUT2

Reason for selecting two node topology instead of the standard 1 node topology
In N7k we have a variety of linecards and we wanted to make sure TAHI passes on all types of line cards. But one of the constraints which we had was
the type of interface on each linecard and the speed of the new linecards which are about to come. If we wanted to do this one way is change the NIC
on Free BSD according to the linecard which we have to test, which is a difficult procedure. Instead of that what we did is, we kept the FreeBSD NIC
as a gigabit ethernet copper port and N7k1 will have a linecard which has copper gigabit port and this setup is permanent. And the same N7k1 will e populated with
different types of line card and that will be connected to UUT via  a trunk port.

Following are the things/requirement which prompted me to write this blog.

  • TAHI on Virtual Machine: Neither in TAHI website nor in anyother sites mention about the issues of running TAHI Selftest Tool on Virtual machines which has become a defacto standard now a days. I will try to put the solutions/workarounds which I had to do inorder to make it work.
  • addr.p2 section issue: I didint want to reboot the device for the addr.p2 section to initate a DAD, as that will cause lot of time. Normally in most of the routers/multi-layer switches admin down and admin up of the ipv6 enabled interface will cause IPv6 DAD to be initiated. This is fairly simple by replacing the reboot command in the reboot_async.rmt script to a interface admin down admin up (More details will follow)
  • Timing Issue in addr.p2 Section: If you want to adminup/down like me for the addr.p2 section, then trhere is another issue which needs to be tackled. Since we are doing a admin down and up in TAHI by passing the control frm TAHI to Vremote, sometimes interface will be up and must have send DAD before the TAHI pkt capture program gets the control and this will cause the TAHI testcase to fail as TAHI wouldn’t get any DAD NS packets from device as it have already send.
  • Alternate Connection via mgmt port instead of Serial port: This was one of the most important thing, because In my case its always difficult to have a serial cable connected to the device just to run TAHI as in our company we use terminal server to connect to devices.
  • Other parameters: I wanted to run TAHI with multiple options for our devices like in Cisco nexus 7000 has got lots of variety of linecards and I wanted to make sure TAHI is passing in any type of linecard. For that I had to change the TAHI default single node topology to a dual node topology

TAHI On Virtual Machines
   When you are running TAHI in a Virtual Machine, following additional settings you need to do on ESX Vsphere

  1. Need to Add a serial port to the VM. This can be done by right clicking on teh Free-BSD VM –>Edit Settings–>hardware Tab–>Add–>Serial Port–>Next–>Select the option Use Physical Serial Port on the Host–>Select the correct physical serial port–>next–>finish
    Note: This is required only if you want to TAHI to run config commands on device via serial port. I didnt want this functionality as I was using management telnet of my device.
  2. Make sure TAHI test links are attached to a Vswitch where no other VMS are connected, else traffic from other VMS may cause issues
  3. Enable Promiscuous mode in Vswitch (This can be done by Home–>Inventory–>Configuration–>networking–>Vswithc–>properties–>Select vswitch–>edit–>security)
  4. Sometimes, in my case, ESX server will send IGMPv3 queries to the Virtual machines. To check this run tcpdump on any of the free bsd interface (tcpdump -i em1) and if there is a IGMPv3 packet coming with SMAC set to null mac then tahts the issue. As per Vmwares website (http://www.vmware.com/files/pdf/technology/esx35_ip_multicast.pdf) it is used for Vmotion, but this can cause problems in TAHI. To disable this go to
    Home–>Inventory–>Configuration–>advanced Settings–>Net (Now go to the bottom part of that list nad from there slowly scroll up to see the following string Net.IGMP Queries (it will be set to 2, change it to 0 to disable it)

Addr.p2 Section Issues
  Since In my case I have opted to go with interface admin down/up instead of reboot for intitating duplicate address detection, theres an issue which we need to tackle. We need to make sure there is a delay in the interface being coming up because, if it comes up so fast, then by the time the v6eval-remote script reboot_async.rmt passes the control to tahi for capturing packets, dad would be already send and testcase will fail. The way I made sure there is a delay was by using the help of Cisco’s EEM script,
In the reboot_async.rmt i made sure only I will do the shut and then the script passes control to TAHI. Internally the EEM script what it will do is , I have a tracking option enabled for the interface which I shut in the reboot_async.rmt in the EEM script, and the action defiuned for that event is to admin up the interface and there is a additional sleep time of 10 seconds also given t make sure enough time is there.

Following is eem config from NXOS
“track 1 interface Vlan10 line-protocol    !!!this will track the interface Vlan10 when it goes down
event manager applet adminup
event track 1 state down               !!Whenever the tracking interface goes down do the following
action 1 cli conf t                          !!go to config mode
action 2 cli int Vlan10                    !!go to interface mode
action 3 cli sleep 10                      !! sleep for 10 seconds
action 4 cli no shut”                      !!admin up the interface
Before trying the option I have tried different other options like playing around with stp, but with those I was not able to get a complete PASS across multiple runs. So finally this helped me in getting complete pass across multiple runs.
Note: Somehow this issue happens once in 3 runs or so, so if you dont want to run continious runs of TAHI, no need for such a tweak.

Alternate Connection via mgmt port instead of Serial port
    Since having a dedicated serial connection to UUT was not possible in our company, we had to look for other options like TAHI to connect via mgmt interface or via  terminal server port. Joseph matress and Sean helped me with this and it proved to be really helpful and faster execution than via serial port. I chose to  connect via mgmt interface than via telnet. Following files needs to be edited for the same, assuming nut.def has beeen already configured properly
/usr/local/v6eval/tn.def  –> Set the RemoteDevice to the Mgmt Ip address of the uut
RemoteDevice 10.104.104.117
/usr/local/lib/perl5/site_perl/5.10.1/V6evalRemote.pm–> In this file change the following entries
In this file here is a section called BEGIN {
In that set the following
$Bye=”exit”;
$CU=”telnet”;
my $TermCmd = “$CU -l $User $Device”;
I still kept  the -l option as I was using the root user to login to freebsd, if you dont use the -l $User, telnet will assume root as the username and try to connect which will fail in ase of NXOS as the NXOS username is admin. so having -l $User is a good idea.

Another issue which I faced while running a batch run of TAHI multiple times was that the passowrd or username which was send through the rLogin procedure in V6evalRemote.pm was having a delay and due to which the login to device fails. The main problem was that the default rLogin procedure was not taking care of the retries correctly eventhough as per code it seems to do so. This was a difficult one for me as this was seen only once in 3 or 4 iterations. I tried changing multiple things and didnt succeed. Finally I replaced the rLogin procedure with a proc which I got from perl expect site which i have pasted below.


sub rLogin($)
{
my($timeout)=@_;
my($retry)=8;
my($i)=0;
my($t)=$init_timeout;
my($is_admin)=0;
my($is_admin2)=0;

     $timeout=50;    # special patch for long login timeout (such as netbsd-i386)

#   if($Remote == undef) {
if(!defined $Remote) {
print STDERR “rOpen() should be called first.\n”;
return 0;
}

    if($debug) {
print STDERR “prompt_user: “$prompt_user{$Type}”, “.
“prompt_password: “$prompt_password{$Type}”, “.
“prompt_command: “$prompt_command{$Type}”\n”;
}
$Remote->expect($timeout,
[
qr’login: $’,
sub {
print STDERR “rLogin: Got Login prompt\n” if $debug;
$spawn_ok = 1;
my $fh = shift;
$fh->send(“$User\n”);
exp_continue;
}
],
[
‘Password: $’,
sub {
print STDERR “rLogin: Got password prompt\n” if $debug;
my $fh = shift;
print $fh “$Password\n”;
exp_continue;
}
],
[
eof =>
sub {
if ($spawn_ok) {
die “ERROR: premature EOF in login.\n”;
} else {
die “ERROR: could not spawn telnet.\n”;
}
}
],
[
timeout =>
sub {
print STDERR “rLogin: Retry error\n”;
print STDERR getOutput();
return 0;
}
],
‘-re’, qr'[#>:] $’, #’ wait for shell prompt, then exit expect
);

    #
# login
#
return 1;
}

Another important thing to note while using alternate management connection is to modify the reboot script. Since by default, TAHI expects the UUT to be connected via the serial port, reboot.rmt script which is part of the vremote package reboots the device and waits for the login prompt to come after reboot. This wont work with management connection as the telnet/ssh connection will be disconnected at the time of switch reload. So the work around for this is to edit the reboot.rmt script in the directory corresponding to your device and change the below

Default reboot.rmt

#Open a connection to the DUT
rOpen() || goto error;
$rOpt_timeout = 300 if ! defined($rOpt_timeout);

#Login to the device
rLogin(5) || goto error;
#reboot the device and wait for the log in prompt
rReboot($rOpt_timeout) || goto error;

### wait for login prompt (race conditioning for NetBSD-sparc)
rLogin($rOpt_timeout) || goto error;
rLogout($rOpt_timeout) || goto error;
###

Tweaked reboot.rmt for alternate mgmt connection

rOpen() || goto error;
$rOpt_timeout = 300 if ! defined($rOpt_timeout);

rLogin(5) || goto error;
rRebootAsync($rOpt_timeout) || goto error;
#Sleep for 200 sec. Change the seconds according to your device’s reboot time.
sleep(200);

rClose();
exit($V6evalRemote::exitPass);

error:
rClose();
exit($V6evalRemote::exitFail);


Other Issues or Common Mistakes which needs to be aware of

  1. Make sure the UUT wont transmit any other type of data or control packets other than the expected ipv6 packets. examples are spanning-tree BPDUs, cdp packets, lldp packets etc. In my case I have disabled spanning tree and cdp to take care. Alternatively for cisco devices you can use bpdufilter for the same. If any such packets are coming which cannot be disabled then can think of blocking those packets via MAC ACLS as well.
  2. Sometimes TAHI/tcpdump wont be able to capture any packets if there is no IPv4 address assigned to that interface and because of this also testcases can FAIL.
    To avoid this assign dummy IPv4 addresses to the FREE BSD interfaces which are connected to the UUT.

Some usefule Tips/quick Reference while running TAHI

  1. Running Phase-1 tests: Change directory to the untared Selftest Tool location and issue the command “make ipv6ready_p1_router”. Replace router with host if you want to test it against hosts. [Refer Makefile.test in the Selftest tool directory]
    DC3-03-bsd# pwd
    /usr/local/www/data/Self_Test_5-0-0
    DC3-03-bsd#
    DC3-03-bsd# make ipv6ready_p1_router

  2. Running Phase-2 tests: Change directory to the untared Selftest Tool location and issue the command “make ipv6ready_p2_router”. Replace router with host if you want to test it against hosts.
    DC3-03-bsd# pwd
    /usr/local/www/data/Self_Test_5-0-0
    DC3-03-bsd#
    DC3-03-bsd# make ipv6ready_p2_router
  3. How to run specific tests in a section [This is a very useful tip] : There are two ways to do this, but I would always prefer the 1st method as its pretty straight forward and also it gives the test result fo PASS/FAIL as well.
    • Using AROPTS: Change directory to the untared Selftesttool location and then go to the section where you want to run sepcific test cases and then issue the following command make ipv6ready_p1_router AROPT=’ -s 8 -e 8 ‘
      where -s is the starting test case id and -e is the ending test case id. if you just want to run a single test case give same value as argument to -s and -e.
      DC3-03-bsd# cd nd.p2/
      DC3-03-bsd# pwd
      /usr/local/www/data/Self_Test_5-0-0/nd.p2
      DC3-03-bsd# make ipv6ready_p1_router AROPT=’ -s 8 -e 12 ‘
      The above command will run testcases from 8till 12
    • Editing the makefile: Another way to do the same but cumbersome procedure is to edit the make file, eitheryou can comment out all the testcases which you dont want to run in the particular section by editing the following file or creating a new file with the describefd below with only the required testcases. File name changes per section
      1. spec.p2 section: INDEX_p1_router [for phase 2 INDEX_p2_router]
      2. nd.p2 section:  ND Section has multiple subsections
        • nd section:INDEX_ND_p1_router [This handles all the neighbor discovery related testcases]
        • Router discovery section: INDEX_RD_p1_router[This handles all the Router discovery related testcases]
        • Redirect Section: INDEX_REDIRECT_p1_router [This handles the Redirect testcases]
        • addr.p2 section: INDEX_P1_ROUTER
        • pmtu.p2 section: INDEX_p2_router
        • icmp.p2 section: INDEX_P1_router Now to run just go to the section and issue make ipb6ready_p1_router
    • Another way is to directly get the command from the previous test results [This can be found from the result logs of a test case. Look for Command line eg:to run it go to the section where the test case is and execute ./IP_Version.seq -pkt ./IP_Version.def -log 2.html
  4. Tweaking the timing/other test parameters: To tweak some of the wait time or other test related settings there is a config.pl which can be edited. Also awareness of these timers will help in debugging why a testcase failed. test cases in TAHi will fail when we dont receive a packet with in a particular time which is defined in these config.pl. Also its not recommended to change the defaults unless it is very necessory.
    1. Global config.pl: This has the global settings for all testcases and can be found in the root directory of Selftest tool. These settings are useful if your device/NUT doesn’t support all the features and according to the settings, some testcases will be skipped so that only the relevant testcases for yoiur NUT will be executed. Since the device which I work on supports everything I used the defaults
      $TRANSMITTING_EREQ = 1; [Nut supports transmitting echo requests]
      $MULTICAST_ROUTING = 1; [Nut supports multicast routing]
      $MTU_CONFIGURATION = 1; [Nut supports MTU configuration]
      $HAS_MULTIPLE_INTERFACES = 1; [Nut has Multiple interfaces]
      where value 0 means not supported or 1 or non-zero means supported
    2. Per Section config.pl: This file has settings which will be related to the section where this file is found and can be found in the section folder
      • spec.p2: config.pl
        1. $wait_dadns = 5; #wait time for DADNS respond to RA.
        2. $wait_after_dadns = 5; # wait time for transmit packet after received DADNS.
        3. $wait_reply = 5; # wait time for echo_reply, icmp_err, …
        4. $exceed_max = 65; # wait time for ICMP Time Exceeded, for Fragmentation test.
        5. $exceed_min = 55; # accept ICMP Time Exceed if this time passed after receiving 1st fragment
        6. $cleanup = “normal”; [# Common Cleanup Procedure.
          # Available actions are:
          # normal: (UNTER CONSTRUCTION)
          # (1) send RA with Router/Prefix Life Time set to zero.
          # (2) send NA with SLL containing a different cached address.
          # (3) Transmit Echo Request. (never respond to NS)
          # reboot:
          # (1) reboot target.
          # (2) sleep $sleep_after_reboot
          # nothing:
          # do nothing. (only sleep $cleanup_interval)
        7. $wait_incomplete = 10; # wait for Target Neighbor Node Cache state transit to INCOMPLETE.
        8. $wait_rebootcmd = 300; # maximum waiting time for putting reboot command to NUT. ONly in case of automated run
        9. $sleep_after_reboot = 10; # sleep time after reboot.
        10. $cleanup_interval = 5; # sleep time in cleanup “nothing” procedure.
      • nd.p2: config.pl
        1. $wait_rebootcmd = 300; # maximum waiting time for putting reboot command to NUT
        2. $sleep_after_reboot = 0; # sleep time after reboot.
        3. Router Configuration Variables
          1. $min_MaxRtrAdvInterval = 4;
          2. $max_MaxRtrAdvInterval = 1800;
          3. $min_MinRtrAdvInterval = 3;
          4. $max_MinRtrAdvInterval = 1350;
          5. $min_AdvLinkMTU = 0;
          6. $max_AdvLinkMTU = 1500;
          7. $min_AdvReachableTime = 0;
          8. $max_AdvReachableTime = 3600000;
          9. $min_AdvRetransTimer = 0;
          10. $max_AdvRetransTimer = 0xffffffff;
          11. $min_AdvCurHopLimit = 0;
          12. $max_AdvCurHopLimit = 255;
          13. $min_AdvDefaultLifetime = 0;
          14. $max_AdvDefaultLifetime = 9000;
          15. $min_AdvValidLifetime = 0;
          16. $max_AdvValidLifetime = 0xffffffff;
          17. $min_AdvPreferredLifetime = 0;
          18. $max_AdvPreferredLifetime = 0xffffffff;
      • addr.p2: config.pl
        1. $wait_dadna = 5; # time between DAD NS and DAD NA
        2. $wait_solna = 5; # time between NS and solicited NA
        3. Implementation depend conditions
          • $wait_rs = 5; # time between receiving DAD for LLA and receiving RS
          • $wait_dadns{‘reboot’} = 130; # time to reboot *set a little bit longer time than actural time to reboot
          • $wait_dadns{‘ra’} = 5; # time to wait for DAD NS for global address after receiving RA *for IPv6 Ready Logo Phase-1 “reboot” and “ra” are enough time to reboot
          • Nut related Variables
            • $DupAddrDetectTransmits = 1; #DupAddrDetectTransmits How many times the target sends DAD NS packets
            • $DADTransmitsGA = “ANY”; #DupAddrDetectTransmits for GA and SLA, If the target perform DAD for GA or SLA, set “YES”, If the target DOES NOT perform DAD for GA or SLA, set “NO”, If you don’t care whether the target perform DAD for GA or SLA, set “ANY”. *for IPv6 Ready Logo Phase-1, use “ANY”
            • $RetransTimerSec = 1; #RetransTimer/1000 [sec], The time between retransmission of NS
        4. $wait_rebootcmd = 120; # maximum waiting time for putting reboot command to NUT
        5. $wait_addrconf_base = 5; # time to actually address assigment after ending DAD
        6. $lla_autoconf = “YES”; # Specify if the target machine configure its link-local address automatically specify “YES” or “NO”
      • pmtu.p2
        1. Setup related Variables
          • $wait_dadns = 5; # wait time for DADNS respond to RA.
          • $wait_after_dadns = 5; # wait time for transmit packet after received DADNS.
        2. Main Variables
          • $wait_reply = 5; # wait time for echo_reply, icmp_err, …
          • $default_mtu = 1500; # default MTU specified by Media Type. default: 1500[octet] (on Ethernet). CAUTION: except 1500 is now unsupported.
          • $tmpdef = “./tmpPacket.def”; # temporary file of packet definition.
        3. Cleanup related variables
          • $cleanup = “normal”; [# Available actions are:
            normal:(1) send RA with Router/Prefix Life Time set to zero.
            (2) send NA with SLL containing a different cached address.
            (3) Transmit Echo Request. (never respond to NS)
            reboot:
            (1) reboot target.
            (2) sleep $sleep_after_reboot
            nothing:do nothing. (only sleep $cleanup_interval)
            ]
          • $wait_incomplete = 10; # wait for Target Neighbor Node Cache state transit to INCOMPLETE.Note: this is only used in cleanup “normal”. DELAY_FIRST_PROBE_TIME = 5 [sec]
            RETRANS_TIMER = 1,000[msec]
            MAX_UNICAST_SOLICIT = 3
            5 + 1.000 * 3 = 8[sec]
          • $cleanup_interval = 5; # sleep time in cleanup “nothing” procedure.
        4. $wait_rebootcmd = 300; # maximum waiting time for putting reboot command to NUT
        5. $sleep_after_reboot = 10; # sleep time after reboot.
      • icmp.p2
        1. $wait_reply = 5; # Time to wait for reply of packet from NUT to packet which TN sent. echo reply , icmp error message, echo request from NUT to TN is also contained
        2. $wait_time_exc = 65; # The maximun time to wait for fragment reassembly time exceeded message
        3. $reboot_incleanup = “NO”; # whether to reboot in cleanup() function, YES: reboot No: to not reboot
        4. $wait_rebootcmd = 300; # maximum waiting time for putting reboot command to NUT
        5. $sleep_after_reboot = 0; # sleep time after reboot.
        6. Cleanup related variables
          • $wait_time_for_ns = 10; # This time use in cleanup() function.This parameter shows time after transmitting na until TN receives ns N times.
          • $count_ns = 3; # This count use in cleanup() function.This parameter shows number of times which receives NS.
Categories: IPV6