Hi Ron,

I still think you will see an increase in route discoveries if the receiver is going GCs, although with the numbers you've chosen ({0, 20, 45}) it's marginal whether every GC will cause a FIFO overflow. I think it would be better to go with {0, 60, 60}. I doubt whether the extra 55mS will make any difference. Route discoveries are expensive; you want as few as possible.

The backoff algorithm is very clearly defined in the spec, and I can't help thinking that the backoff delays and exponents were chosen for a good reason. And if SPOTs use a different algorithm to other 802.15.4 devices there might not be equitable network usage (assuming it ever becomes important that SPOTs work in the same environment as other 802.15.4 devices).

I'm sure 800mS is fine for a 4 hop transmission (8 hops really including the hops to get the end-to-end ack back) provided there aren't any/many link-level retransmissions. But as soon as there are, perhaps because of collisions in a busy environment or because the signal strength is marginal, I predict things will go crazy, with lots of unnecessary radiostream retransmissions. Since the hop limit is 7, and since your experiment suggests you need 200mS of timeout for each hop, surely the timeout has to be at least 1400mS? If you allow for say, 1 link-level retransmission for every 7 hops (i.e. 2 link-level retransmissions for the round trip) then you'll need a end-to-end timeout of about 1600mS. I can't see any case for it being less than that. But clearly, 15 seconds was a bit silly :-).

That's a good improvement.

Yes I imagine so, and it certainly explains why the crc error and channel access error counts went down.

If I understand the surface graph in Marcus Bestehorn's Symposium presentation correctly, the real pain with the Blue stack begins once the size is over 40kb. Judging from their graph, I suggest you try a suite of at least 60kb.

Ron, I really think it's time to create a proper test rig that allows repeatable and consistent multihop testing. I discussed the idea with Pete and Bob more than a year ago, and Bob seemed ready to create a hardware set up with SPOTs connected via coax instead of aerials, so that the set of receivers can be accurately controlled. If you had, say, a matrix of 16 SPOTs with controllable pathways you'd be able to conduct some very meaningful experiments. A simpler form of that rig would be very useful for Eric, too, for the continuous test system.

--John

==== Original message From: Ron Goldman <Ron....@sun.com> Date: 04 March 2009 Time: 11:15:29 AM Subject: radio update

---- John,

Some more background. Before doing anything I did some informal experiments, running a SPOT app that just displayed the CRC errors, channel access failures & rx fifo overflows in its LEDs. Then I tried doing some OTA commands over several hops and looked at the errors involved.

What I quickly discovered was a huge number of CRC errors & even more channel access errors. So with my changes I tried to understand the causes of those errors and make changes to decrease them.

Let me try to answer some of your questions.....

-- Ron --

On Mar 2, 2009, at 11:43 PM, John Daniels wrote:

My thinking here was basically not to try to worry about why a SPOT might not respond, but to instead try to more quickly have the attempt to transmit complete, one way or another. If a SPOT didn't reply because it was doing a GC then tough---radio packets are inherently unreliable and it is up to higher layers of the radio stack, e.g. Radiostream, to recover from dropped packets.

I have not tried to measure the number of NO_ACKs, but I have not seen any increase in the number of route discoveries. Of course the SPOT app I am running is not causing any GCs....

Since I was seeing many, many channel access failures my first fix was to increase the number of attempts to send and that did seem to help. My reading of the 802.15.4 spec was that it is okay to choose any valid value, so we don't always need to use the default one.

One change I just made is to change the initial backoff exponent to 0 so that there is no backoff delay before the first attempt to send. I'll be trying that out tomorrow to see if it makes any noticeable difference. I'm also thinking of modifying the retry part of the CSMA- CA algorithm so that the backoff exponent for the first retry is immediately jumped to 3 (since delays for a value of 1 or 2 are going to be less than the time to transmit a full packet) and then incremented by 1 for any subsequent retries. So technically that won't be exactly according to the spec, but I think that change will not break compatibility with any other 802.15.4 compliant receiver.

As I mentioned the old radiostream timeout was 15 seconds, which explained why I was seeing the OTA commands fail while waiting for a CRC level ack. As soon as I lowered the radiostream timeout below 1 second those errors just went away. At one point I tried using a 400 millisecond timeout, but that was a bit too short. Right now I am using an 800 millisecond timeout and that seems to be a good compromise. Deploying an app over 4 hops had very few retransmissions.

Note the retransmission now isn't scheduled until after the transmit happens, so any route finding happens outside of the end-to-end timeout.

The major reason for the channel access failures I think can be explained by the lowpan layer trying to send the next packet before the previous one has moved out of radio range. I.e. if A is sending several packets to D with B & C acting as relays, then after sending a packet to B, A must wait long enough for B to relay it to C and also for C to send it out to D. Otherwise A's next packet will collide with the relayed packet. IF A & C send simultaneously (because they cannot hear each others transmission), then B will not receive the packet from A (the hidden terminal problem).

After each lowpan packet I have the sending thread wait for 10 msecs plus another 10 msecs if it was a broadcast packet or plus (5 + 7 * number of hops) for a meshed packet.

After sending a RREP or RRER routing packet add a 10 msec delay.

I presume the added delay is the reason for the decrease in FIFO overflows.

Adding the delays by far had the most effect.

Well the good news is that multihop transmissions now work much better. The bad news is that adding the delays does slow down the single hop tests:

Time to send 20000 integers = 23863msecs, target was 12400 RadiostreamMasterTest test 1: Expected: 12400 Got: 23863

Time to send 500 integers = 12662msecs, target was 11800 RadiogramMasterTest test 6: Expected: 11800 Got: 12662

We sent 1000 integers = 27260msecs, slave received 983 target is: 150 Time to send 1000 integers = 27260msecs, target was 6400 RadiogramMasterTest test 7: Expected: 6400 Got: 27260

in Isolate:

Time to send 20000 integers = 28349msecs, target was 19200 RadiostreamMasterTest test 1: Expected: 19200 Got: 28349

Time to send 500 integers = 17676msecs, target was 15350 RadiogramMasterTest test 6: Expected: 15350 Got: 17676

We sent 1000 integers = 31968msecs, slave received 985 target is: 150 Time to send 1000 integers = 31968msecs, target was 11700 RadiogramMasterTest test 7: Expected: 11700 Got: 31968

Right now I'm focused on adding enough delays to make OTA over multiple hops work reliably. Later we can try to minimize those delays to improve throughput.

802-15-4-systemtests & MultihopBaseTests

I used the BounceDemo, which is 25652 bytes.

not yet

Me too. The more I examine matters, the more amazed I am that the previous radio stack worked at all...

p.s. I forgot to mention in my previous message that before I made any changes I could not reliably do OTA or deploy an app over more than 2 hops.