I have what might be a sort of fun issue for you guys to ponder. I'm not really asking for advice, although I'm curious as to what you might think about the situation.
A customer installed one of our boxes in their facility. The box stopped working properly a couple of days ago. They sent the box to us, and it's working without any issues at all here in our networking environment.
When it was out there, if I used a remote login service to connect to the desktop of the box immediately after the box booted up, I could log into it. Otherwise, I couldn't access the box at all.
When I was finally able to get into the box, I noticed our services couldn't be restarted. Upon start, they would generate an error message indicating that the box itself was out of networking resources.
Have you ever seen that before? Personally, I've never observed that problem on a box. I guess I've always been in networked environments that were properly configured and designed.
None of our other boxes out there have this kind of problem, but admittedly, this box is kind of special, in that we have to have a single port made available to the outside world. Not a big deal, really... just port-forward to our box, and everything is peachy-keen. Nobody else has a problem doing that... but I suspect these guys did something "different"... since he was concerned that port-forwarding was a security issue. He felt more secure just exposing the entire box to the outside world and dropping our internal firewall.
(Note: The box doesn't appear to be infested, in case you're wondering... virus scans find nothing on it).
One way to try and figure out what's going on is to download and install WireShark onto the box. WireShark is a packet capture and analysis tool.
Since you're accessing the box remotely, you'll have to filter out the packets associated with your remote control session, but that would definitely show you if the box is being attacked.
Spell
At the moment, the box is safely in our own facility.
But someone out there would need to investigate it, not me.
I think your assessment, though, is spot on. Someone is hammering the box from outside their network, and they aren't handling it properly. Likely, the problem will go away if they just put a firewall in place and forward the port we want.
Heh... so we can repair it from the damage of improperly securing the
network?
well I was just thinking that you could see if that was the problem that way.
DoS Binder
The plot thickens.
Yesterday through today, the spare box we sent them worked well. The box that they returned to us showed absolutely no problems, so we shipped it back without changing anything on it (except to ensure nobody could modify the executables).
They swapped the two boxes out, and now the box they returned to us is dead on their network again.
One difference between the two boxes: the spare we sent didn't have an a/v capture card, so it didn't make sense to install all the bits that allow it to stream a/v content to stenographers... which is the one port that needs to be exposed to the internet.
I'm starting to wonder if the network engineer was forwarding the port in some bizarre way that floods the box.
I also wonder if he opted to use a MAC address to forward stuff... such that the other box wouldn't have been a problem anyway (because it didn't have the right MAC address).
In the end, I told them that the box worked flawlessly in our facility, and that the problem must therefore be with their network in some way. When they said the other box worked fine, I told them I couldn't explain why (which, technically, I can't... I have no idea how they have their network set up). I then told him that I didn't particularly care what their network policy was, or how they have their network environment set up... I am only responsible for ensuring our equipment works properly, and it does.
We'll see if they ever figure it out. In the meantime, their customer is going berserk. I wonder if they'll lose the account. I wonder if someone out there will get fired for this. I wonder who that person will be (the network engineer who screwed up their network and is protected by the CEO, or his boss, who can't fire him).
Well, after weeks of this kind of nonsense, it gets harder not to point out the nature of our working relationship. They've been abusing the hell out of our technical support contract, and we don't appreciate it.
But now since IBM owned everything, it was ALL their problem and they had to fix.
Wed Jul 21 2010 11:11:56 AM EDT from Ford II @ Uncensoredwe had the best time when ibm bought one of our vendors. It used to be that they'd pawn problems off on each other saying the problem lied with the other guy.
But now since IBM owned everything, it was ALL their problem and they had to fix.
Filenet?
well, at $work we run into MTU problems now and then.
indicators are rather clear that its misconfiguration on our side. though our admin team doesn't seem to be able to track down the issue. (I sometimes get the feeling they don't want to)
me just sees that the replies are clamped..
and if I lower the mtu of my workstation to 1400 everything is fine.
first time I saw the vpn-device had some absurd mtu, and that was it.
second time they just wanted to show me the webinterface of their smoothwall (or whatever it was, some fedora embedded cruft) and were offended as I told them I wouldn't believe what some webinterface prints, just ifconfig counts.
This time I just requested changing the mtu on the vmware instance, and easily got away without any discussion.