atom feed12 messages in org.freebsd.freebsd-stableRe: igb on a Nehalem system, buildwor...
FromSent OnAttachments
Mars G MiroJan 8, 2009 6:16 am 
Jack VogelJan 8, 2009 8:44 am 
Mars G MiroJan 8, 2009 10:19 am 
Mars G MiroJan 8, 2009 10:50 am 
Jack VogelJan 8, 2009 11:06 am 
Mars G MiroJan 8, 2009 11:16 am 
Christian BruefferJan 8, 2009 2:00 pm 
Mars G MiroJan 9, 2009 12:02 am 
Jack VogelJan 9, 2009 1:15 am 
Mars G MiroJan 9, 2009 2:14 am 
Mars G MiroJan 9, 2009 2:22 am 
Mars G MiroJan 12, 2009 3:21 am 
Subject:Re: igb on a Nehalem system, buildworld stats
From:Mars G Miro (sp@anarchy.in.the.ph)
Date:Jan 8, 2009 10:50:45 am
List:org.freebsd.freebsd-stable

On Fri, Jan 9, 2009 at 2:33 AM, Jack Vogel <jfvo@gmail.com> wrote:

Well, I am at Intel you know, and even we don't seem to have any systems with 82576 down in my group here. The way link works I can be about 99.9% sure in saying its not the driver. Its preproduction so there are lots of possibilities, and the biggest problem is its going to be difficult to help when I don't have any such hardware :(

I've heard from the 1G product team that they have seen EEPROM mismatches on systems that will result in things not working in funny ways.

Jahh, I've seen those but not w/ Intel NICs. I believe it was from Broadcom on some IBM x3455? (IIRC) and it was indeed quite amusing ;-)

If you have a back to back connection to another NIC on Port 0, no switch, does it still autoneg to 100?

I will have do that tomorrow as I am @home now ;-)

btw, another data point, during sysinstall, we encountered:

<unknown network interface type> on both the igbs.

Thanks.

On Thu, Jan 8, 2009 at 10:19 AM, Mars G Miro <sp@anarchy.in.the.ph> wrote:

On Fri, Jan 9, 2009 at 12:44 AM, Jack Vogel <jfvo@gmail.com> wrote:

I have not seen a problem like this ever, what is the link partner of each NIC and if you switch the ports what happens?

Hi Jack,

They're connected to a GigE switch. It was just one w/ the first NIC, but having seen that it only connects at 100baseTX, I wired the 2nd and saw that it can now do 1000baseTX. Unfortunately w/ problems as it can 'see' some machines but unable to see others (in the same physical network segment). I've changed cables, and plugged them in different ports in the switch but still the same behavior.

IIRC, this is the first time I had igb problems and only on this box. I believe I encountered igb NICs in the newer HP DL380/385 but those work fine.

btw, this is a Supermicro Intel Engineering sample box (major vendors don't have Nehalems in the market yet) so there prolly are hardware/driver bugs lurking? I dunno.

Thanks.

We have Nehalem's in the validation lab but I have not had an excuse to install on one so far, I guess now I do :)

On Thu, Jan 8, 2009 at 6:16 AM, Mars G Miro <sp@anarchy.in.the.ph> wrote:

Hi guys,

I just got on my hands today a NEHALEM system:

2 x 5560 Nehalem CPU (2.8GHz, 8MB cache memory, 6.4GT/sec [QPI]) 12GB 1333Mhz DDR3 Memory 1 x 500GB SATA HDD

FreeBSD 7.1-RELEASE/amd64 install fine, however I seemed to be having problems w/ its built-in Intel NICs:

igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4> ether 00:30:48:c5:db:e2 inet6 fe80::230:48ff:fec5:dbe2%igb0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect (100baseTX <full-duplex>) status: active igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4> ether 00:30:48:c5:db:e3 inet6 fe80::230:48ff:fec5:dbe3%igb1 prefixlen 64 scopeid 0x2 inet 172.17.32.32 netmask 0xffff0000 broadcast 172.17.255.255 media: Ethernet autoselect (1000baseTX <full-duplex>) status: active

The first NIC would always want 100baseTX no matter how I'd ifconfig down/up it, so I just had to use the 2nd NIC. Unfortunately, this too is having problems. Like being unable to 'see' some machines on the same network segment. Some other machines are accessible. And yes I've double-checked the network stuff (cables, switch, IP settings) and my conclusion is b0rky NICs.

pciconf -lvc: igb0@pci0:1:0:0: class=0x020000 card=0x10c915d9 chip=0x10c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks cap 11[70] = MSI-X supports 10 messages in map 0x1c enabled cap 10[a0] = PCI-Express 2 endpoint igb1@pci0:1:0:1: class=0x020000 card=0x10c915d9 chip=0x10c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks cap 11[70] = MSI-X supports 10 messages in map 0x1c enabled cap 10[a0] = PCI-Express 2 endpoint

So anyone else having igb problems? I'm downloading 200812-CURRENT now (is tehre gonna be a 200901-CURRENT ISO soon? :-p), I'd like to try that, but checking cvs seem only a handful of changes.

Also I did some buildworlds: make -j8 buildworld 2846.900u 2266.188s 15:50.43 537.9% 6375+2082k 10084+7937io 1482pf+0w make -j16 buildworld 3518.254u 2175.593s 14:23.29 659.5% 6656+2147k 26165+8546io 4300pf+0w make -j32 buildworld 3582.897u 4437.710s 18:03.88 739.9% 6528+2125k 5725+7930io 1555pf+0w

Verbose dmesg: http://pastebin.com/f5f799561