nRF52 Port and fixes

Porting to nRF52 has been done for the nRF OpenMesh code and ble_gateway example. Added a bandwidth test project to the set of examples.

Fixed issues :

• Memory issues that prevented the bootloader from running correctly after reset
• Issues #128, #134 , #135 and #130 are fixed

Known issues:

1. Side by Side DFU with signing is broken
2. gcc build of bootloader is broken
3. Bandwidth test application does not support nRF52
4. Standalone bootloader is broken
5. Bandwidth test for slave without ack is broken
6. Mesh blinky target is not present in the Keil UV4 build

Reset issue fixed and porting to nRF52

All the devices should need to be reset before the new application successfully get started for v0.8.5. This release fixes this.

Fixes

• Few memory setting is done so that the new application now gets started without any reset .
• Porting to nRF52 has been done

Known Issues

• gcc build of bootloader is broken

Cleanup of debug code

Some debug code was left around in the timer.c module by mistake for v0.8.4,
affecting performance. v0.8.5 fixes this.

Fixes

• Removed debug code causing the timer module to fail regularly, affecting
  performance.

Side by side DFU

This release brings in concurrent DFU operation with the application. By
allowing the DFU functionality in the bootloader to be called from application,
we can now transfer firmware updates while the application is running. The API
for Application side DFU is contained in /nRF51/rbc_mesh/dfu_app.h, and the
feature can be enabled by adding /nRF51/rbc_mesh/dfu_app.c, mesh_flash.c and
nrf_flash.c, then adding compiler define MESH_DFU. The memory requirements for
the bootloader changes for this release, please take a look at the bootloader
readme for more information.

All permutations of the bootloader has been precompiled and included in
/nRF51/bootloader/bin/.

New features

• Side by side DFU.
• Packet peek has been included in the main API, allowing easier access to
  the scanner.
• Dual access addresses: The framework always receives on both the standard BLE
  access address, as well as an application specified mesh-access address.
• Structured parameters for all events ("Softdevice style" .params in each
  event)
• µs-level timestamp on all RX and TX events.

Fixes

• Fix for issues #94, #95, #98, #99, #101, #102, #109, #111 and #117.
• Timestamps all relative to global epoch, instead of per timeslot.

Bugfixes, code restructure and merge to master

After a long time releasing to the sdk-8-support branch, this release is moving development back
into master. This means that we're moving the old master branch into a new legacy-sdk-6-7 branch, to
allow those who are heavily invested in the old SDKs to continue using them. As with the old
master, this legacy branch will not see new features. The sdk-8-support branch has been merged to
master, and will no longer be present in the repository.

This release targets a series of issues reported since 0.8.2, in addition to a superficial
change to the cache system: We've taken the caches out of the version_handler module into its own
module, named handle_storage. The handle storage acts as a pure storage module, serving the
version_handler. The version_handler (and it alone) manipulates the handle storage.

New features

• The bootloader now yields TX events after successful transmits, one per channel transmitted on.
• Added a bootloader_verify.py script under nRF51/bootloader/pc-util/, to ease debugging of any
  unintended bootloader behavior.

Fixes

• Fix for Issues #72, #74, #75, #78, #79, #80, #82.
• Fix for example case in Issue #91, still following up on this one.
• CRC false-positives should no longer occur. Symptoms included unused handles getting data and
  erronous payloads getting through to the application.
• DFU signature verification reboot-loophole closed.
• Fixed some erronous DFU packet lengths.
• Restructure of version_handler -> new module handle_storage encapsulating caches.
• Removed Bootloader journaling function, replaced by full check verification.
• Fix GCC linker script for Bootloader.
• Improved documentation for Bootloader usage.

Name change, Bootloader stability and RX reporting

This release marks a name change for the project: you are now looking at the release notes of the
nRF OpenMesh! Don't worry though, the URLs, APIs, structure and path-names remain the same.
We've also addressed some important issues and feature requests, both with the core-part of the
mesh framework and the DFU bootloader. Take a look:

New features:

• RSSI now added to all RX-related events, as requested in Issue #66 and #70.
• Advertisement address from which the RX came from reported. Note that this advertisement address
  is per hop, and does not identify the device that originated the value update.
• Packet peek: The Transport control module now has a function to set a callback for packet
  reception. The callback gets parameter mesh_packet_t*, which can be used to read adv-address and
  payload in the packet. In addition, CRC value, RSSI and timestamp in microseconds is given to the
  callback.
• Bootloader handles relaying of serial-transfers not intended for them properly.
• Bootloaders are more helpful to neighbors in distress, and will attempt to resend responses to data
  requests which go unanswered.
• A set of pre-merged hex files are added under nRF51/bootloader/test/full_test/ for easy testing of DFU functionality.

Fixes:

• Bootloader had some relay issues, which have been adressed. Performance and success rate for over
  the air-updates should be improved significantly.
• Bootup glitches in the UART-interface removed.
• All zombie states and hang-issues with the core mesh should be fixed.
• Bootloader fails gracefully if the device page isn't flashed.
• nRFgo Studio caused some issues during DFU reset procedure, and has been purged from quick start
  guide. A warning about this is instead present in the guide.
• Some unused files made their way into the public repo, and have been removed.

Stability improvement

This release is a pure stability patch, and should mitigate some issues observed during testing:

• When subjected to stress, the firmware sometimes shortcuts the handle cache linked list, excluding some handles from the pool of handles being transmitted. This is observable in the ping-pong example project if run with more than 10 devices: Devices stop getting updates from the central device one by one, until there is only one device left getting pings on its pongs. This was caused by a race condition, as the version handler had a hole in its thread safety. All manipulation of the cache-linked list is now serialized with the event-handler. This removed all symptoms from all our test cases, and the mesh performs much better under pressure.
• The trickle timers for each handles had a possiblity of falling behind, leading to devices not being able to catch up with its timers. This issue only appeared when the mesh was tested under heavy stress (20+ devices within range of each other), and looked like non-stop spamming of packets. This issue is now mitigated, as the trickle timer is forced into a "future" timeout if the device catches up with a whole interval.
• The bootloader "early-exit" check was moved to after the init-process, as the journal module had to be initialized to make sure that we weren't in the middle of a transaction.

Device Firmware Update over the Mesh

Standalone Bootloader that supports Serial(UART) and Mesh (does not require the softdevice)
PC tool support for DFU over the Mesh (https://github.com/NordicSemiconductor/pc-nrfutil/tree/mesh_dfu)

Changes:
Transport layer for the DFU packets has modified behaviour, however this is restricted to the bootloader
and does not impact any applications.

Known Issues:

• Nodes running the Mesh in application mode will not relay the DFU packets.
• Nodes running the Mesh in DFU/Bootloader mode will not relay application packets.
  (Workaround: Switch all nodes to bootloader mode when doing DFU, it is possible to
  Use the bootloader mode as a relay without actually updating local firmware)
• DFU protocol used does not adapt well to high density networks (when a node can see 8 or more nodes).
  (Improvements are planned for the DFU protocol)

Issues fixed:

• Issue #58
• Issue #62

Features to be added:

• Local Nodes to contribute their firmware to fulfil DFU requests from neighbours.
• Update the "Device Info Page" using the DFU

Bugfixes and stability improvements

This update should greatly improve stability:

• Removing several sources of hardfaults, mostly related to timeslots.
• Dropping updates to the database if an event can't be propagated, so that the update can be processed later, when the event queue becomes available.
• Removing an overflow on mesh payload-search of packets without payload (this issue would make it look like the mesh gets updates to handle numbers you've never seen before).
• Enforcing limit on application-handles - all handles over 0xFFEF are reserved system handles for future mesh maintenance and the upcoming DFU feature.
• Increased default packet pool size to allow all queues to fill completely without overflowing.
• Moved all default size-#defines to rbc_mesh.h, making it easier to configure memory to fit your applications.
• Fix for Issue #52.

So no new functionality for this one, but this release fixes all crashes and any odd behavior we've been able to identify in our tests since v0.7.0.

16bit handles, new GATT interface, async events

The v0.7.0 is the largest update to the bcast-mesh since the initial release almost a year ago.
Bringing several fundamental changes to the core functionality of the mesh, the update should allow for
new usage scenarios and be able to provide better support the existing applications, without forcing
too many big changes. The biggest changes coming with v0.7.0 are:

16bit handles and handle subsets

Based on feedback from mesh-users, we saw that one of the most significant restrictions in the framework
was the number of available handles, and the issues related to scaling the handle space. Up until now,
the bcast mesh has been enforcing a hard limit of 155 handles in a mesh-network, but in practice, we've seen
that both bandwith and memory restrictions have resulted in significant performance problems with
as little as 50 handles.

To combat these problems with scaling, v0.7.0 introduces two major changes to the handle-value system:

• We've extended the handle range from 155 to 65535 handles - using 16bit handles.
• Each device now only keeps track of and rebroadcasts a subset of the handles in the mesh

The subset of handles is managed by two caches: the handle cache and the data cache.
The handle cache keeps track of the version number of each handle, and allows the device
to decide whether an incoming handle value packet is new or old. The data cache keeps
track of the retransmissions by storing the current data for each handle, and and timing
related parameters for that particular handle (the Trickle instance).

The handle cache entries are significantly smaller (in memory) than the data cache entries,
and are also more important for correct behavior. Because of this, the handle cache has to be
larger than the data cache (enforced at compile time). The most recently updated handle
cache entries contain a link to a data cache entry, holding the retransmission data for that
handle. As the data cache fills up and overflows, the least recently updated handles
are discarded first, and the "oldest" handles stop retransmitting. Similarly, the least
recently updated handles are the first to be discarded from the handle cache. This is
analogous to classic LRU-caching schemes.

As the cache sizes may be configured by the application (by overriding the RBC_MESH_DATA_CACHE_ENTRIES
and RBC_MESH_HANDLE_CACHE_ENTRIES #defines in rbc_mesh.h in your compiler), the memory and
bandwidth usage can be controlled by the application. For applications utilizing a low number of
handles, the mesh will behave as it always have, as the cache may fit all values in all devices.

While the cache typically follows the LRU-scheme, there is an option to override this behavior.
By setting the "persistent" flag of a handle, that particular handle may never fall out of the
cache (both handle and data cache). It is strongly recommended that a device that intends to
update a value in the future keeps that value as persistent in their cache, as an update to a
value that the device doesn't know the version of is likely to be suppressed by neighbor devices
which keep the original, higher version number. It is also important that the cache is sized to
handle all the persistent values.

If the application attempts to read values that are no longer present in the cache, the call
always returns with NRF_ERROR_NOT_FOUND.

Serial interface update

As a response to the changes in handle count, the mesh serial interface has been updated
to fit the new format. The arduino-implementation of the application controller has also
been updated to fit these changes. See the
serial interface documentation for details.

Spec-conformant packet format

The mesh has been assigned the 16bit Service UUID 0xFEE4 for this release, moving away from the
previous 128bit UUID. With this feature, the mesh is able to use a BLE-core spec compliant message
format, a feature that has been employed for v0.7.0. The mesh packets now use a proper
<AD-len - AD-type - data> structure, as defined by the GAP specification. The AD-type used is
the "Service data" (0x16), with the service UUID being 0xFEE4. While this adds some overhead to
the packets (and reduces payload size), we think it's a valuable addition, as the mesh data may
be read from any Bluetooth 4.0 compliant scanner, and regular advertisers may inject packets
without any changes to link-layer firmware. While we still recommend using the GATT interface
for accessing the mesh from Smartphones or other applications, this opens up possiblities
for any device to be an active part of the mesh. Read more about the packet format in the
"how it works"-document.

New GATT interface

As the number of handles grew, the GATT interface had to change. The Mesh service will no longer
contain a single characteristic per handle-value, but rather just one characteristic for data
access. This new characteristic follows a specific format, and acts as a two-way
transport medium for mesh access. The GATT handling module now has a new name as well,
mesh_gatt. Read more about the syntax of the new mesh characteristic in the
"how it works"-document.

Async event handling

We've seen some performance issues coming from the way events are given to the application.
The main problem is that by sending them inline as a callback to an event handler function,
the mesh-context is blocked for an unknown amount of time, leading to overflowing buffers,
poor bandwidth utilization and unexpected behavior. To change this, we're moving to
asynchonous event passing; events are now queued up in a FIFO-manner from the framework,
and the application has to pop them off the event queue with the rbc_mesh_evt_get()
function. In the examples, this is done in the main while-loop, in combination with the
Softdevice sleep function sd_app_evt_wait(). This methodology is similar to the
way the Softdevice passes events, and we think it improves overall consistency.

Zero-copy for mesh packets

The final major change is the way packet data is handled internally. Instead of creating several
copies of the packet memory for the internal module, and in addition force the application to
do a copy of any data they want to keep, the framework now passes the same data around, and never
duplicates memory. This includes the application, and this improvement has one additional implication to the
way you have to handle events. To let the mesh-framework be able to know when it may safely free
packet memory for other purposes, the application is required to call
rbc_mesh_packet_release(uint8_t* p_data) with the data pointer in the mesh-event as a parameter
after it is finished processing the event. Failure to do so will result in a NRF_ERROR_NO_MEM
event from the framework to the app_error_handler() callback. The release-function will accept
any p_data from the mesh (including NULL), and we recommend calling this for all events,
regardless of event type.

This change includes removing the mesh memory from the GATT server alltogether, and there
is no longer any need for adjusting linker-maps or heap-size if you want extensive amounts
of handles; only the aforementioned #defines for cache sizes.

Misc changes

There are some additional minor changes:

• The issue #44 hotfix has been pushed into the sdk-8-branch.
• Fix for issue #45
• TX events are now posted after the mesh has transmitted the message, and contains a pointer to
  the transmitted data.
• The rbc_mesh_init function now has a lfclk-field, in which you should supply the same clock-parameter
  as given to the sd_softdevice_enable-function (or SOFTDEVICE_HANDLER_INIT if you're using the softdevice-handler).
  This helps the mesh adjust for clock drift when calculating timeslot lengths.