Update: We resolved the issue permanently and I understand what was going on, but have little time to post everything soon. If you're in a hurry and want help, leave a comment or email me.
One of our customers purchased two BladeCenter H Chassis for deploying VMware on them. Each chassis came with two Nortel switches providing 2 internal ports and 9 external ports. The external network ports are: 3x 10Gbit and 6x 1Gbit.
The customer has a physically isolated DMZ network, so one 1Gbit port from each switch was dedicated to a DMZ switch (VLAN1).
The client had purchased only 2x 10Gbit SFPs, so the third port was empty and won't be used in this setup.
To make use of VMware's Virtual Switch Tagging (VST) network concept, the switches which the blades have to be connected to must be configured as a trunk and allow the required VLANs to pass. Also, the Nortel switch on the BladeCenter must be configured to pass those VLANs, for both external and internal ports.
The following VLANs were created:
One of our customers purchased two BladeCenter H Chassis for deploying VMware on them. Each chassis came with two Nortel switches providing 2 internal ports and 9 external ports. The external network ports are: 3x 10Gbit and 6x 1Gbit.
The customer has a physically isolated DMZ network, so one 1Gbit port from each switch was dedicated to a DMZ switch (VLAN1).
The client had purchased only 2x 10Gbit SFPs, so the third port was empty and won't be used in this setup.
To make use of VMware's Virtual Switch Tagging (VST) network concept, the switches which the blades have to be connected to must be configured as a trunk and allow the required VLANs to pass. Also, the Nortel switch on the BladeCenter must be configured to pass those VLANs, for both external and internal ports.
The following VLANs were created:
- Management
- vMotion
- Fault Tolerance (FT)
- Virtual Machines
After configuring the external and internal ports of the Nortel switch to be part of those VLANs, a strange problem popped up: I couldn't ping or reach any of the ESXi hosts in any way, unless I pinged my workstation from within the ESXi server first!
To make matters clear, here's how things were connected:
My workstation -> Server Farm Switch
BladeCenter -> Server Farm Switch
Ping from workstation to any ESXi host: Fails
After 1 ping from an ESXi host to my workstation: Succeeds, and all pings from my workstation to that specific ESXi host go through.
Also, even after traffic is established and I connect using vSphere Client, it disconnects me after about 15 minutes and I can no longer communicate with that host until I ping my workstation from that host again!
After poking around for hours, the solution was to take one external port (1Gbit) out of all VLANs except 1. That is, that port must not belong to any VLAN except VLAN1 (untagged). Doing so, allowed us to communicate with all servers smoothly.
I still don't understand why that worked and whether traffic is now passing through the tagged external ports or that specific untagged port. I'll be doing more investigations next week and update this post.
Hi,
ReplyDeleteI have similar case:
BCH with 1 Nortel Network Module and 1 IBM network module.
The Blade are HS22 with 2 NIC.
I install ESXi 4.1 on them.
I don't use VLAN at all.
Problem:
1. The VM can get IP from DHCP, but it cannot ping outside. If I ping it from outside, it works, but not always.
2. The workaround that seems to be working is to add another NIC to that VM. But users reports the VM may have the same problem again if it reboots. (lost connectivity).
Additional info:
The Cisco Switch where the Nortel network module connected reports that EXT5 and EXT6 are blocked because of some network violation (but sadly it doesn't say what it is).
So, I disable EXT5 and EXT6 from the Nortel Management menu. But it doesn't seem to be working consistently.
Right now, I don't know what else to do. I'm thinking of throw away the Nortel network module and replace it with the IBM module, but this will create downtime.
Any clue?
Thank you so much in advance for your help.
Fajar Priyanto,
ReplyDeleteFor some reason I can't see your comment here but I got it as an email. I hope you can see/receive my reply.
Make sure that the Default PVID for the Internal Ports (only internal) is set to 1.
Let me know how things progress with you.
Also, do you plan on using EtherChannel?
Damn.. did I just lose my comment? :)
ReplyDeleteI have similar case:
Blade Center H, with 1 Nortel Network Module and 1 IBM Network Module.
The blades are HS22 with 2 NIC.
I don't use VLAN at all.
I install VMware ESXi 4.1 on them.
Status:
- Each blade can be ping from other machine
Problem:
- VM installed on them can get DHCP IP, but it cannot ping other machine.
- If I ping the VM from other machine, it will work and VM can ping other. But not persistently. After sometimes it will stop again.
Temporary solution:
Adding another network card to the VM seems to be working but not always, especially on Vista VM.
Additional Info:
EXT5 and 6 on the Nortel Network Module got blocked by the Cisco Switch because of network violation (but Cisco doesn't say what the violation is). So, I disable EXT5 and 6 from Nortel Management Menu, but doesn't seem solve the problem.
I'm thinking of throwing away the Nortel Module and just use the IBM 2nd Network module, but this will result downtime.
Any clue? Thank you so much in advance.
Thanks I've retyped my comment from memory :)
ReplyDeleteLet me check your clue
I thought Etherchannel is activated by default if we use Switch Network Module (like this Nortel).
ReplyDeleteThere are two ends to a connection:
ReplyDelete[client side] <---> [blade side]
The client side has to enable EtherChannel on their side and then you have to configure "Trunks" on Nortel.
Trunks on Nortel are Cisco's EtherChannel.
Make sure you configure the Cisco side to use RAW EtherChannel. Do not configure LACP.
Ok this is the info:
ReplyDeleteAll INT and EXT port PVID: 1
All INT : VLAN tagging enabled
All EXT : VLAN tagging disabled
Is there something wrong?
From what I understand from your requirements, your network is flat (no VLANs to be used at all).
ReplyDeleteIn that case, make sure the default PVID is 1.
What that option does is the following:
If you set the number to 105, for example, then in ESX you create a Port Group without a VLAN ID, all the traffic from that Port Group will be sent untagged, correct?
Nortel will auto-tag it with VLAN 105.
So if your network is flat (no VLANs), make sure you set the Default PVID to 1.
The external ports would have it on 1 by default. If not, put them all on default PVID 1.
Yeah, disable VLAN tagging on the internal ports.
ReplyDeleteYou either tag on both ends, or you don't at all.
If I disable VLAN tagging on INT, will it disrupt the current network activity?
ReplyDeleteIf you have configured Port Groups that use VLANs on ESX, then you should enable VLAN tagging on both internal and external, and also on the client side (Cisco switches).
ReplyDeleteYou can change the settings on one internal port (for one blade server) and test things there.
Make sure you always apply the new settings.
Save only after you're done. If you do NOT save, you can always revert back to the previous settings.
Ok I've disable VLAN tagging on all INT (except MGT. should I also disable it on MGT?).
ReplyDeleteI have applied and save.
So far looks good. It doesn't seem to disrupt the current network operation.
I'll follow up with users if they still experience network problem on the VM.
I'll report back in the comment soon.
Thank you so much MBH ^^
Do not touch the MGMT ports. These are to manage the Nortel itself.
ReplyDeleteIf this is a production environment, I highly encourage you to advise them to use VLANs.
vMotion & FT should be separated on their own VLANs to not have any traffic interrupting them.
Without VLANs, if a virus/worm breaks loose, it'll cause everything to stop. With VLANs, you can block traffic from certain VLANs in such situations.
You can keep your current flat network and add VLAN configuration. If you do, make sure the Cisco switches are configured to use VLAN Tagging & the nortel switches are configured to enable VLAN tagging on both the internal and external ports.
Thanks for the insight for VLAN.
ReplyDeleteI'll learn more about it.
Btw, currently the Statistic on EXT1 is much higher than other EXT ports (although they are increasing too).
Does it mean that the blades are only / mostly using EXT1 for traffic? Is this bad?
Can we make it balance across EXT ports?
Make sure EtherChannel is configured properly.
ReplyDeleteIf you just connect the cables, traffic won't be distributed. EtherChannel (Cisco) / Trunking (Nortel) has to be configured on both ends.
The problem still persists. Intermittently.
ReplyDeleteI'll try to update the ESXi 4.1.
If it's still like that, I'll remove the Nortel network module.
Hi again!
ReplyDeleteToo bad! I thought it got resolved. Can you draw me the network infrastructure setup and email it to me? Both logical and physical.
Also, send me the output of the Nortel switch configuration (dump).
Hi,
ReplyDeleteI have a very annoying and confusing issue that is very similar.
I have 5 HS21 blades insiden IBM E-Series enclosure with a single IBM Nortel BNT L2/3 Switch which has 14 internal GB ports and 6 external GB ports.
Now I have no VLans and this is all on a flat network with no tagging etc setup.
I have Windows installed on blade 1 and RedHat Linux on Blade 2 and there working fine.
BUT...
I have installed ESX4.1 onto the 3 remaining blades with default install (no VLAN setup so its 0).
I cannot connect to the blades and they cannot connect out to anything including each other.
The switch config for all ports is
INT1-5 (Win, Linux, ESX blades)
Tagging Enabled
PVID Tagging Disable
VLan ID 1
EXT1 (Connected to 12port GB dumb switch)
Tagging Enabled
PVID Disable
VLan ID 1
So I can connect to and from the first 2 blades fine, but can't do anything from ESX.
Can ANYONE help with this please?
Thanks,
Mark
Mark,
ReplyDeleteHello there. Try disabling VLAN tagging in the Nortel switch for the ESX blade ports.
If the Default PVID is not 1, set it to 1.
Let me know how it goes for you.
Hi,
ReplyDeleteI have disabled VLan tagging already and left PVID at 1.
So tried:
Tagging Disabled
PVID Tagging Disabled
PVID 1
Also:
Tagging Enabled
PVID Tagging Enabled
PVID 1
Also:
Tagging Enabled
PVID Tagging Disabled
PVID 1
With these configurations I have set the vSwithch to VLAN "Unset" (0) and also Vlan ID 1.
But no matter what I try as above it will not allow any packets in or out... any other suggestions?
Thanks,
Mark
also I swapped the blades between bays just incase it was a faulty port (highly unlikely but you never know)
ReplyDeleteMark,
ReplyDeleteLeaving it untagged is the correct approach, unless you plan to add VLANs in the future.
How is the Nortel switch connected to the customer network, and how are you connected to the customer network?
Is trunking enabled on the customer switches? EtherChannel?
Hi,
ReplyDeleteat the minute the external network is not part of the design.
We are just trying to get the 5 blades all talking together.
So we have Windows 2008 on the first blade configured in the same subnet as the 3 ESX blades, we also have Linux installed in Blade 2 with the same subnet.
Now Windows and Linux and talk to each other fine, but nothing can tyalk to the ESX blades and they can't talk to anything.
I have my workstation currently connected into the AMM module for management of the system, so all tarffic is just within the blade, once we get that working we'' be looking at setting up a trunk to the production network.
I am at a loss as from my previous post I have tried the various configurations and nothing seems to work.
It has to be something with the Nortel and VMware vSwitch I think, but is there anything else you can suggest?
Thanks,
Mark
Mark,
ReplyDeleteI wasn't successful in setting Nortel's vswitch as it gave me more problems than solutions, so I kept it disabled in my implementations.
Are you pinging the windows hosts from the config (yellow) screen of ESXi, or are you pinging from the service console (enable it in the Troubleshoot menu)?
In ESXi, how many vNICs have you added to the management network? The default is vmnic0 only. Try to keep only that NIC connected and uncheck the other ones.
It's possible that VMware is trying to ping through the other NIC, and if your blades don't have IPs on the 2nd NIC, they won't be reached.
Hi,
ReplyDeleteNot sure what you mean by Nortel vSwitch.
I have one NIC attached to ESX (confirmed vmnic0 connected)
It is attached to INT3 of theNortel with tagging and PVID disabled and PVID of 1
I have one NIC attached to Windows with Tagging and PVID Tagging disabled and PVID set to 1.
I am trying to ping the service console IP and also ping windows from the service console (using ALT+F1) to access it.
I have to be missing something obvious and it should "just work" in this very basis setup.
Anything else you need me to verify?
Thanks,
Mark
Mark,
ReplyDeleteI was talking about Nortel's vCenter switch connectivity, and mistakenly called it Nortel vSwitch. It allows the Nortel to connect to vCenter and pull the VLAN configuration.
Anyway, try this: Login to the 2nd Nortel switch (lower one) and disable the Windows, Linux & ESXi internal ports. This way you make sure that Windows/Linux aren't seeing the lower switch as the primary, and ESXi is seeing the upper switch as primary.
I hope you've deployed ESXi & not ESX.
Hi,
ReplyDeleteAh ok didn't even know such a thing existed.
I actually only have 1 Nortel switch installed (top).
I also installed the latest firmware in an attempt to rule that out.
Thanks,
Mark
And yes its ESX4i
ReplyDeleteTry connecting your workstation directly to the Nortel switch and ping all the servers (windows, linux & esxi).
ReplyDeleteHi,
ReplyDeleteSame result.
I have ping Windows and Linux no problem, I get nothing from ESX.
I have installed ESX3.5 and ESX5 to test, same result, I booted one of the ESX blades with a Linux Live CD and can ping it fine.
So its only when ESX is installed I can't connect, so the HW is good.
Just doesn't add up now!
Thanks,
Mark
Mark,
ReplyDeleteTry swapping the IPs of the ESXi with the Windows/Linux boxes.
I lost the Windows IP when I put it on ESX.
ReplyDeleteWhen i put the ESX IP on Windows it started working after about 20secs.
Swap the esx blade location with the windows one.
ReplyDeleteIf that doesn't help, try resetting esx to default settings & set nothing apart from the IP.
Hi,
ReplyDeleteAlready done the swapping of blades previously, had no affect.
Also I have installed ESX and different versions with nothing configured apart from IP and same result.
Reading online there seems to be plenty of isses around the IBM Nortel L2/3 switch and ESX but so far nothing has worked...
Its gonna be a long weekend.
Could you try setting up a static arp route in the nortel switch?
ReplyDeleteI was actually trying that but when I do and ping the IP the workstation shows the MAC as all zero in its local cache.
ReplyDeleteDo you have any other switch at hand? A Nortel L7, a Brocade or a Cisco?
ReplyDeleteNope,
ReplyDeleteOnly have 2 of these Nortel L2/3 switches (same thing on both).
I have added a new VLan to the switch (Vlan99) and setup it on the 3 internal ports of the ESX servers and also in the port group of the Service Console and all 3 servers can ping each other.
So they have Tagging Enabled, PVID Tagging Disable and a PVID of 1.
The Service Console has a VLAn ID 99 and they can all connect to each other, but not to anything else (which would make sence).
Confused!
Perhaps the native VLAN (defaults to 1) doesn't play well on Nortel.
ReplyDeleteTry setting the default PVID of windows & Linux to 99. Does it see the esx boxes?
Nope
ReplyDeleteI just get the vlan in windows using BACS utility didnt work, also in Linux using NetworkMan.
So I then set the PVID of the port to 99 and that also didn't work.
They still can't ping ESX!!!
Mark,
ReplyDeleteI'd open a support case with the Nortel people.
I haven't tried setting up a blade center without an upstream router, so in all my deployments, we were connected to server-farm switches/routers and servers could see each other.
I think I'm out of ideas :/ sorry to not being able to help fix this thing.
Hi,
ReplyDeleteNo problem at all thanks for all your time.
It is very confusing as the settings are all right and it should work, I have done amny setups in similar configs with HP and DELL, never had this type of issue!
Thanks Again,
Mark
I've done the exact setup many times with the same hardware, but I think the switches our guys sell are L7 not just L2/L3.
ReplyDeleteDrop me your email (you can find my email in my profile by clicking on my nick) and I'll check with our guys tomorrow.
I use VMWare VST. How did you permanently resolved the issue ?
ReplyDeleteI can't understand how sending my VLANs to VMWare.
Thanks
Gilles,
ReplyDeleteRead the comments written here, as I've explained them to the guys before.
If you still face problems, let me know.
I've read all the posts. But I can't understand why it doesn't work.
ReplyDeleteAnd I'd like to know what PVID tagging is.
I can post my dump if you want.
I have two VLAN working, and i can't understand why i can't make a third one working.
The story :
ReplyDeleteIn a Bladecenter H, there is two "BNT Layer 2/3".
On both, Ports EXT5 and EXT6 are a trunk to an external switch (HP Procurve).
The following configuration works well for VLAN 12 and 2 but I've just add 33 and this last one doesn't work.
I've added an interface with IP in this VLAN. I can ping it from an ip interface in HP Procurve (both ways) but i can't ping a VM or from a VM in this VLAN.
--> One of the 2 BNT conf (the one with the ip in VLAN 33) :
script start "BNT Layer 2/3 Copper Gigabit Ethernet Switch Module for IBM BladeCenter" 5 /**** DO NOT EDIT THIS LINE!
/* Configuration dump taken 15:55:22 Fri Jan 6, 2012
/* Version 1.5.15, Base MAC address 00:18:b1:61:19:00
/c/sys/timezone 194 /* Europe/France
/c/sys/dlight e
/c/port INT1
pvid 12
/c/port INT2
pvid 12
/c/port INT3
pvid 12
/c/port INT4
pvid 12
/c/port INT5
pvid 12
/c/port INT6
pvid 12
/c/port INT7
pvid 12
/c/port INT8
pvid 12
/c/port INT9
pvid 12
/c/port INT10
pvid 12
/c/port INT11
pvid 12
/c/port INT12
pvid 12
/c/port INT13
pvid 12
/c/port INT14
pvid 12
/c/port EXT1
pvid 12
/c/port EXT2
pvid 2
/c/port EXT3
pvid 12
/c/port EXT4
pvid 33
/c/port EXT5
tag ena
pvid 12
/c/port EXT6
tag ena
pvid 12
/c/l2/vlan 1
def INT1 INT2 INT3 INT4 INT5 INT6 INT7 INT8 INT9 INT10 INT11 INT12 INT13 INT14
/c/l2/vlan 2
ena
name "dmz"
def INT5 INT6 INT7 INT9 INT12 INT13 EXT2 EXT5 EXT6
/c/l2/vlan 12
ena
name "VLAN_LAN_2"
def INT1 INT2 INT3 INT4 INT5 INT6 INT7 INT8 INT9 INT10 INT11 INT12 INT13 INT14 EXT1 EXT3 EXT5 EXT6
/c/l2/vlan 33
ena
name "VLAN_SRVTOIP"
def INT1 INT2 INT3 INT4 INT5 INT6 INT7 INT8 INT9 INT10 INT11 INT12 INT13 INT14 EXT4 EXT5 EXT6
/c/l2/vlan 4095
def INT1 INT2 INT3 INT4 INT5 INT6 INT7 INT9 INT11 INT12 INT13 INT14 MGT1 MGT2
/c/l2/stg 1/off
/c/l2/stg 1/clear
/c/l2/stg 1/add 1 2 12 33
/c/l2/stg 2/off
/c/l2/trunk 1
ena
add EXT5
add EXT6
/c/l3/if 33
ena
addr 172.18.33.202
mask 255.255.255.0
broad 172.18.33.255
vlan 33
relay disabled
/c/l3/dns
prima 172.18.98.121
secon 172.18.98.122
dname "anfh.fr"
/c/sys/ntp
on
prisrv 172.18.98.121
secsrv 172.18.98.122
/
script end /**** DO NOT EDIT THIS LINE!
Gilles,
ReplyDeletePVID when set, sends the packets of that VLAN without requiring a tag from the blade server; i.e., If PVID is 12, packets sent from VMware that are untagged, will be tagged as VLAN12 by the BNT switch.
For that reason, I set the PVID on internal ports only to something that the client won't use (example: 3999), and then configure the VLANs on VMware (VST). I keep the PVID on external ports on BNT set to 1.
When configuring VLANs, you should configure them in 3 places: VMware port group, BNT and the external switch (HP in your case).
So on the BNT side, you add your external ports to the VLANs you created, and on the HP side, you trunk the ports to all the VLANs you want to pass over (or all VLANs, since BNT will accept only the defined ones).
Trunk between HP et BNT switches works. I can ping from one to an other in my new VLAN 33.
ReplyDeleteThe problem is that I can't ping machines in Port Group with VLAN ID 33. It works for the others VLAN.
Gilles,
ReplyDeleteI never configure an interface with an IP (L3) on BNT because there's no need for it.
You cannot reach the VMware servers because of tagging problems.
Change the PVID of internal ports to 3999, then change the PVID of external ports to 1.
Then create the VLANs in L2 configuration, and add the external ports to the VLANs. An external port can belong to one or more VLANs.
Thank you for your help.
ReplyDeleteThe problem was ... the VM.
I created a new VM Network-booted on WinPE. I 'pinged' and it just worked.
My previous testing machine seems to be network failing.
Thanks again.
Gilles
Gilles,
ReplyDeleteI'm glad it's working. I still don't think you should use your existing BNT configuration.
You should also create a trunk group (PortChannel in Cisco language) for your external ports , then create a failover trigger.
This allows you to use Link Status Failure Detection in VMware rather than Beacon Probing.
You can create a trunk group out of 1 external port.
We have 1 External port as a trunk and have passed various VLANs on it, basically a trunk port.
ReplyDeleteIt is possible that we trunk all internal ports connected to that switch as a trunk with that external ports?
Right now when i go on and try to trunk the INTERNAL port with the external port i get the following error:
Port EXT1 and INT4 in Trunk group 1 should be members of the same VLAN
Feb 29 9:25:03 10.1.34.226 ERROR cfg: Error: Ports INT4 and EXT1 in Trunk group 1 have different port types
so is it possible to trunk the internal port with the external port?
We have 1 External port as a trunk and have passed various VLANs on it, basically a trunk port.
ReplyDeleteIt is possible that we trunk all internal ports connected to that switch as a trunk with that external ports?
Right now when i go on and try to trunk the INTERNAL port with the external port i get the following error:
Port EXT1 and INT4 in Trunk group 1 should be members of the same VLAN
Feb 29 9:25:03 10.1.34.226 ERROR cfg: Error: Ports INT4 and EXT1 in Trunk group 1 have different port types
so is it possible to trunk the internal port with the external port?
Bilal,
ReplyDeleteYou do not put the internal NICs in the trunk group because it doesn't make sense. Each BNT switch will have 1 NIC connected to it only per blade server.
All you have to do is this:
- Create VLANs
- Add external & internal ports to VLANs
- Create trunk group
- Add external port(s) to trunk group
The trunking of traffic (traffic aggregation) will occur between the customer side (Cisco/Juniper) and the external BNT ports. There's no need to include the internal ports, because the packets will be delivered to the internal port normally (it strips the packets of the trunking part of it).
I hope this makes it clear.
MBH,
ReplyDeleteThanks for the reply. We have done implemented all the 4 points that you have mentioned, i.e.
- Create VLANs
- Add external & internal ports to VLANs
- Create trunk group
- Add external port(s) to trunk group
My query here is that Can i assign multiple VLAN's to a single INTERNAL port?
Bilal,
ReplyDeleteOh of course you can! All you need to do is enable VLAN tagging for each internal port then add the internal ports to the VLAN groups you created.
MGH,
ReplyDeleteI have tried to do that, Under Layer 2, when i add that particular INTERNAL port to all the VLANS it accepts.
But then again, Under the switchports tab, when i click on that particular INTERNAL port it asks me for Default Port VLAN ID (1 - 4094).
What should i put in here? I have tried to put in a value 1, which is default vlan of my switch, but on the blade end i am not able to ping any of the vlans..
Bilal,
ReplyDeleteYou shouldn't need to change the Default PVID.
If you're using VMware, you need to configure the VLAN ID on the management NIC, if you're trying to ping from the console.
If you're connected to the host directly, create a few port groups on the vSwitch to match your VLAN IDs on the BNT switch.
If you're not using VMware and have installed Windows directly, then you cannot use multiple VLANs unless some sort of 3rd party software to do that. Windows does not tag packets for VLANs natively.
Linux has a package that handles packet tagging, but I'm not sure if it can handle multiple VLANs or not.
Which one is your case?
Thanks a bunch. It worked:)
ReplyDeleteNow can you tell me that how can i import the configurations of the BNT switch and import it on another switch?
Basically i am configuring it for failover...
By the way
ReplyDeletei am using VMware ESX 4.1 :)
Bilal,
ReplyDeleteI'm glad it worked out for you, and I hope you mean ESXi not ESX, because ESX shouldn't be used anymore.
If you're using the BNT's web UI, there's a button/link on the top called "Dump" -- that will show the switch's config.
Save it into a text file, then run an ftp/tftp server and go to the other switch's System settings, Config, then enter the ftp/tftp info, the name of the config file then press Get Config.
Make sure your BNT switches' firmware is up to date. The last config I'm aware of is v6.7. -- if you have 1.x, you need to upgrade to 5.x then 6.x.
MGH,
ReplyDeleteThanks for the reply. Can you pls tell me on the command line? All the steps which i have to follow to copy the current configuration of the switch and paste it onto the other switch?
Thanks for your help :)
Bilal,
ReplyDeleteI don't know the commands in the CLI but you can refer to the application guide: http://www.bladenetwork.net/userfiles/file/G8124_AG_6-7.pdf
And by the way, my nick is MBH not MGH ;)
This comment has been removed by the author.
ReplyDeleteHello out there , has anyone had an issue with IBM HS22 blade dropping their internal connections on and off to a 6 port and 10 port BNT switch ?
ReplyDeleteWe have 8 of these chassis H here with all of them having the same issues.
Kingtut,
ReplyDeleteYes, it happens if you sometimes keep the PVID at 1, and the customer's native VLAN is not 1.
What I usually do is set the PVID for the internal ports to a VLAN that customers won't use (3999), enable VLAN tagging & add the internal & external ports to the required VLAN.
I seem to still have the internal port dropping from the blade to the BNT switch even after changing the default internal vlan to 3999. Its kind of odd where as its only effecting blade 10. and not the other blades right now.
ReplyDeleteThis is what I see:
Jun 5 6:25:45 ENCL-1-BNT-7 NOTICE server: link down on port INT10
Jun 5 6:25:46 ENCL-1-BNT-7 NOTICE server: link up on port INT10
Jun 5 6:27:21 ENCL-1-BNT-7 NOTICE server: link down on port INT10
Jun 5 6:27:22 ENCL-1-BNT-7 NOTICE server: link up on port INT10
Jun 5 6:27:22 ENCL-1-BNT-7 NOTICE server: link down on port INT10
Ideas?
Kingtut,
ReplyDeleteDoes that blade have the same configuration as other blades? Same exact config? (BNT, OS, Software)
Run the hardware diagnostic tools & see what shows up. It may be a hardware issue with the blade's NICs.
Both BNT switches are exactly the same firmware etc .. The OS on my blade that is having issues IS different than the other 14 blades in the chassis. Q: why would an OS potentially cause an internal port go down and come back up ? the DSA comes back with no issues with the hardware. The firmware versions are exactly the same across this chassis for all blades.
ReplyDeleteAnonymous/Kingtut,
ReplyDeleteIf you're running Windows, try installing Broadcom Advanced Control Suite and use NIC teaming to create a logical adapter and assign an IP to that adapter.
Did you enable failover triggers? Make sure the ports are properly setup there.
You should put each VLAN in its own STG and enable PVST+ support.
Dump your config into pastebin.com and link it here and I'll take a look.
Dear Sir, pls help.
ReplyDeleteI'm a new IT admin for this Bladecenter HS23 - I have a problem when I try to build a vsphere system (5 ESXi on 5 blade)
In this Bladecenter - I have [Blade 01] install ESXi6.7, on [blade 01] have [VM1]->[Port group vlanID:10]
- I have [Blade 02] install ESXi6.7, on [blade 02] have [VM2]->[Port group vlanID:10]
Switch network is: IBM Networking OS Virtual Fabric 10Gb Switch Module for IBM BladeCenter - I had config VLAN tagging and add vlan 10 on INT1 & INT2, but VM1 can't ping to VM2. But if [Port group] on vSwitch of ESXi not tag vlan (vlanID:0) then VM1 can ping to VM2.
I want to use about 5 vlan in ESXi, So please guide me.
newbie,
ReplyDeleteYou need to enable VLAN tagging on the internal as well as external ports of the blade chassis switches. Once you enable VLAN tagging, and assign the VLANs to the INTA/EXT ports, then you can create port groups in VMware and do the VLAN tagging from the portgroups.
Please keep in mind that vSphere 6.7 end of support is coming soon, so you'll probably need to move to vSphere 7 soon, if it's supported on the servers you have.