Ethernet fails connecting after a while

The EtherTen combines an Uno-equivalent Arduino-compatible board and Wiznet-based Ethernet support, along with a microSD card slot and Power-over-Ethernet support. [Product page]
Sleurhutje
Posts: 28
Joined: Thu Mar 01, 2012 9:18 am

Re: Ethernet fails connecting after a while

Post by Sleurhutje » Wed Apr 18, 2012 8:56 pm

The use of printable characters has to do with DNS registration during the DHCP sequence. Some characters are not valid for host registration and will DHCP to.fail or result in unpredictable behaviour. To be sure the hostname is valid, just use printable characters. The last three bytes of the MAC address are concatinated to the text WIZ.

When a connection is forced every two seconds, changes are real the previous connection isn't completed and the socket is still in à WAIT state. When you start a new connection at this point, the buffer is not fully cleared but new data adds up. This will probably result in an overflow in the IP stack or your input buffer of the Arduino. In advance to crashing things might slow down first.


A little demonstration:
- Open a command prompt/DOS box
- Enter "netstat -an" without the quotes
- Copy the results of thee TCP connections
- Open your webbrowser and enter your URL
- Now redo the "netstat -an" at the command prompt

You'll see your server's IP and probably a CLOSE_WAIT state as the status. Now redo the "netstat -an" every few seconds. After time the idle state will end and the connection, and corresponding socket, is cleared. Play with this, also with several active browser windows simultaneous (causes multiple but individual connections to the same server). Maybe this will add a little more of understanding TCP connections. Having only a few sockets and limited buffers available is very hard setting up complex connection orientated communication. UDP is faster but a connectionless protocol without handshaking do data might get lost during transmission.

More info on the state of the connection can be found here: http://commandwindows.com/netstat.htm

Scargill
Posts: 8
Joined: Tue Apr 17, 2012 12:26 pm

Re: Ethernet fails connecting after a while

Post by Scargill » Wed Apr 18, 2012 11:33 pm

Thanks - REALLY - for your input that's really appreciated as I'm only starting to get to grips with this..

Ok, in slightly simpler English... the first bit about DNS makes absolute sense - just to clarify - you're saying only the last THREE digits need to be printable?

So you're saying if I poll every too seconds I'm pushing it. What would be a more reasonable poll - remembering I want to use this to transfer info to and from a database as near to real time as poss at the PHP end.... 3 seconds? 5 seconds? 10 seconds???

Scargill
Posts: 8
Joined: Tue Apr 17, 2012 12:26 pm

Re: Ethernet fails connecting after a while

Post by Scargill » Wed Apr 18, 2012 11:35 pm

One more question - your test you recommend.. didn't follow the middle bit "copy the results of 3 tcp connections..."

Sleurhutje
Posts: 28
Joined: Thu Mar 01, 2012 9:18 am

Re: Ethernet fails connecting after a while

Post by Sleurhutje » Fri Apr 20, 2012 8:21 pm

Scargill wrote:you're saying only the last THREE digits need to be printable?
The last 3 bytes are put behing the text "WIZ" and will be used as the host name for the network. When you use DHCP, the host name is parsed to the DNS server. When you receive a valid IP-address you can PING your interface using both IP-address and host name. E.g. is your MAC address is like 0x01,0x3a,0xf8,0xb1,0x41,0x30 the host name would be like "WIZ▒A0". That's not a valid host name! It might cause problems on networks. So the easiest way is to use what they call printable characters. The best way is to change the library not to use the last three bytes of the MAC address but use a seperate variable for the host name.

More about host names: http://tools.ietf.org/html/rfc952
Scargill wrote:So you're saying if I poll every too seconds I'm pushing it. What would be a more reasonable poll - remembering I want to use this to transfer info to and from a database as near to real time as poss at the PHP end.... 3 seconds? 5 seconds? 10 seconds???
Polling a webserver this way isn't the correct methode. A webserver that lost it's cache can take a few seconds to recompile the script. If you really need to update that fast, create a client/server connection based on TCP. Each communication is checked by handshaking and rock solid. If lossing some data isn't a problem, you can create a client/server connection based on UDP. Just connect to the server, transmit your data and close the connection. Don't care if the server did receive the data. :mrgreen:
Scargill wrote:One more question - your test you recommend.. didn't follow the middle bit "copy the results of 3 tcp connections..."
Just to compare the last state with the original state. ;)

Scargill
Posts: 8
Joined: Tue Apr 17, 2012 12:26 pm

Re: Ethernet fails connecting after a while

Post by Scargill » Sat Apr 21, 2012 3:23 pm

Thanks very much - your explanation makes the MAC issue perfectly clear.

On client/server you assume too much.. :) Could you point me to any working example of a cleint/server connection based on tcp/ip... I just want to send a package and receive a package constantly... I tried all sorts with that client polling but eventually no matter what, it slowed down to a halt.. might take hours, but eventually it would stop.

Sleurhutje
Posts: 28
Joined: Thu Mar 01, 2012 9:18 am

Re: Ethernet fails connecting after a while

Post by Sleurhutje » Sat Apr 21, 2012 9:33 pm

Client/Server communication based on TCP or UDP requires also programming in C#/C++, Visual Basic, Java, Delphi or any other high-end language to create the server side. That application needs to listen to a specific port for a connection by using sockets. Data to be transfered needs to ben read from a source like database or (text)file. Handling of disconnects and broken connections needs some additional checking etc. But that's the only way to process data in a way faster than one connection per 30 seconds. This 30 seconds is the average time-out for incorrect connections. My sketch runs best at a delay of 90 seconds or more.

Sleurhutje
Posts: 28
Joined: Thu Mar 01, 2012 9:18 am

Re: Ethernet fails connecting after a while

Post by Sleurhutje » Mon Apr 23, 2012 7:43 pm

Discovered today that the serial port is the cause of corrupt ethernet data. Especially on high baud rates. When the serial port is set to high speeds, the IP configuration gets corrupted, data is transferred incorrectly and erratic behaviour is a fact. Even if there's nothing attached to the serial port. When setting the baud rate to something of 19200bps or 9600bps things go much better.

My sketc uses an ANSI VT-100 Terminal emulation for displaying the data on the computer. Over and over it failed after adding Twitter functionality. Even converting values to strings failed. Back from 115200bps to 19200bps and works fine.

My screen on the terminal emulation:
Image

Twitter account used for some primative logging (needs much fine tuning) is @wonderzoek.

It logs temperature (DHT21 & LM35), relative humidity (DHT21), gas detection (MQ7), PIR motion detection, gets it's time from the internet or otherwise from a DS1307 RTC, uses DHCP, can use an optional LCD display through I2C. Logs are send to a webserver on the internet. And sends Twitter message from now on.

Sleurhutje
Posts: 28
Joined: Thu Mar 01, 2012 9:18 am

Re: Ethernet fails connecting after a while

Post by Sleurhutje » Thu May 24, 2012 9:07 am

Found something interesting that speeds up the disconnecting in some cases. There is an option in the HTTP/1.1 header fields that tells proxy servers to disconnect immediately after the connection has finished it's task. Some more references can be found here in RFC2616, section 14.10.

The part with the GET command

Code: Select all

client.println("GET /myscript.php HTTP/1.1");
client.println("Host: myserver.mydomain.com");
client.println("User-Agent: Arduino-Sample");
I've added the second line of code below to the sketch

Code: Select all

client.println("Connection: close");

dtbaker
Posts: 26
Joined: Sat Jun 02, 2012 12:52 pm

Re: Ethernet fails connecting after a while

Post by dtbaker » Sat Jun 16, 2012 8:29 am

Experiencing failed connections after my etherten runs for anywhere between 30mins to 1hour. It does a HTTP request every 2 minutes and eventually just fails on the client.connect()

Will implement some ideas here and see if it helps.

dtbaker
Posts: 26
Joined: Sat Jun 02, 2012 12:52 pm

Re: Ethernet fails connecting after a while

Post by dtbaker » Sat Jun 16, 2012 11:58 am

Well those ethernet fixes are already in arduino 1.0.1 gui libraries.

I've just tried changing HTTP/1.0 to HTTP/1.1 with the connection: close to see if that helps.

also put a delay around the client.close();

* waits *

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests