Issues with Linksys Routers 4/16/07 and 5/15/07

4/16/07

The Public2, Staff2, and Meetings Routers began having problems on 4/16/07. Routing works, but wireless does not.

Psycho Space Puppet tried our spare Linksys, but he had the same problem after flashing the firmware to dd-wrt. It worked until that point. He was not able to roll back to the original factory firmware. All of this problem went away at 5:45 PM, or appeared to for the time being. We do not know why at this point. Possible causes of this issue:

-A noxious Windows Update affecting all the clients. This is very unlikely since we don't have the patron machines update until Monday eve, and this started Monday morn.

-Channel (or other) interference. We're noticing more wireless networks showing up in the vicinity, presumably from neighbors.

-Someone physically damaging the routers (farfetched, but we're considering everything.) The return to normalcy at 5:45 seems to refute this.

-Some manner of corruption or hacking of the firmware on all these routers. How would this happen so quickly on the "spare" one though? And it doesn't appear it could be permanent, since things went back to normal at 5:45

-Compatibility problems with our upstream hardware.

-An electrical surge or somesuch sent through our network. Seems unlikely since nothing else was apparently affected, and we should be protected from that anyway.

-A patron laptop or some device was connected to the patron network and either a virus or other pernicious piece of software was attacking points on our network from there, including the LinkSys and the ExpressTerm PCs. When this machine left the building, everything reverted to normal before the attack. This doesn't explain the Staff2 problem unless the laptop virus was attacking wireless points through wardriving, in which case, why did it spare the Orinoco routers?

-County IT secretly planted a chip in Psycho Space Puppet's body which interferes with our wireless routers when he is in the building. (I think we've eliminated this one, since PSP is here and it's working!)

-Interference from another kind of device (electrical company, telephone).

-A temporary, but especially noxious, infection on a library-owned patron machine, which attacked the system until the reboot automatically restored the old image..

-Other, unknown factors.

We are still looking at this. All the non-Linksys routers were working, and those serve most machines so this wasn't stopping us cold.

Results from NetStumbler:

Meetings is on channel 1 along with 2 other outside networks that are floating in. However, it consistently showed on the list, unlike the other 2.

Staff2 showed intermittently.

Public2 didn't show at all, even if one is standing right over it with a laptop.

Another, possibly related issue: APAC2 and Expressterms 1 and 2 were down because of network problems, though they connect to one of the non-Linksys routers. All other machines connected to that router seem OK. The point in repair or "connect to network" that they stop is "renewing IP address" if not sooner. They are not static IPs so that shouldn't be an issue. Is the Public access point somehow overwhelmed?

Note: for APAC2 at least, it initially says that it failed to connect to "Public2" then it doesn't connect to "Public" either when you try.

5/15/07

The exact same problem recurred on 5/15/07. We identified a laptop as a "suspect" and we asked the owner if he could shut it down. At the same time, we restarted many patron PCs. As soon as the laptop owner left the building (he had to go right after he shut his laptop down for us), the problem went away completely.

We strongly believe the origin of this problem is some kind of malevolent or faulty piece of software on this laptop or that is temporarily downloaded to a public computer (and wiped with reboot by Disk Protection), or a faulty piece of hardware on this laptop or the laptop owner's person (like a walkie-talkie interfering with the signal).

If the laptop owner returns to the library, and this problem recurs, we should try to help the owner see if the machine has a viral infection or other problem. If this problem recurs without the laptop owner, we will be fairly certain it is caused by an issue effectuated on one of the public PCs.

In this case, if it was a public PC, it was almost certainly in the "express term circle" or the YA area, due to the circumstances and the timing of the reboots.