Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Should sensor firmware version be moved to a new field? #55

Open
matdon17 opened this issue May 11, 2021 · 10 comments
Open

Should sensor firmware version be moved to a new field? #55

matdon17 opened this issue May 11, 2021 · 10 comments
Assignees
@apswong
Labels
admt approved documentation Improvements or additions to documentation priority priority topic R27

Comments

@matdon17
Copy link

There is concern about whether the current practice of including sensor entries for each firmware version is the correct way to manage sensor metadata in reference table 27.

The main example of this are variants of the Sea-Bird CTD which currently has the following:

  • SBE41, plus 3 entries with specific versions
  • SBE41CP, plus 23 entries with specific versions
  • SBE41N, plus 3 entries with specific versions
  • Versions of the SBE41_IDO (with integrated dissolved oxygen sensor)
  • SBE61, plus 6 entries with specific versions

However, there are also isolated examples that might include similar versioning information which may or may not be significant:

  • MP40_C_2000_G
  • QUARTZDYNE_DSB301-10-C85

These two are recent additions to the Google Drive document:
https://docs.google.com/spreadsheets/d/1Aw8B7FFUjG4e9MveD5qqvI7ZbB-z_x32IQnpHZbXVZA/edit#gid=6
And are captured along with the KELLER_20SX at: #127

To add to complexity, there are now SUNA_V2 and ISUS_V3 for spectrophotometers, although this is likely a genuine change in sensor design rather than just a firmware change. @vpaba @SOCCOM-BGCArgo perhaps you know more on this already?

@mbelbeoch has relayed the possibility that the version numbers for Sea-Bird CTDs may be in someway related to customer batches, but that this is unconfirmed.

The suggestion has been made by @apswong that the firmware version of the sensor could be moved out of the SENSOR_MODEL variable in the NetCDF (and R27 in the process), to a new variable such as "SENSOR_MODEL_REVISION_NUMBER" in which the version number can be captured. For this to happen R27 would need to be tidied-up, and the DACs would need to re-write the meta files. As an intermediate step, "SENSOR_MODEL_REVISION_NUMBER" (of something like that) could be introduced as an optional variable to avoid proliferation of new sensor models.

I think there are a number of other factors to consider with this suggestion before proceeding:

  • similar NetCDF fields like FIRMWARE_VERSION and MANUAL_VERISON are unconstrained fields and as such are of limited value due to the various different ways in which they are populated and the content is open to interpretation;
  • the addition of a separate firmware/revision number field would need to be managed to ensure it is useful and avoids generating an unconstrained field, which is of both low utility to a user and potentially storing up a separate management problem for DACs (at some point will there be a need to standardise this field?);
  • what do current versions really mean (predominantly firmware version it seems) - we want to avoid ambiguity as to what such a field could/should include;
  • it might be conceptually trivial to implement for the SBE41/SBE61, but how would the field be used for other sensors (e.g. BGC sensors)?
  • could we end up needing another controlled vocabulary to make sense of the versions, which might start to look like the current reference table 27 in the process?

It would be good to attempt to capture why the versions started to be added before making changes;

  • was this in some way to capture generations of floats in-lieu of reliable serial numbers and knowledge of batches?
  • or were there significant differences in early versions of SBE41s that were worth capturing at the time but which in later versions maybe aren't important?
  • or did we just fall into using it because the version information was available to us?
  • in the extreme: if there is literally no value to this version information is it worth the effort of managing it at all?

Broadly speaking, the additions to vocabularies need to be well controlled to ensure the consistency of the vocabulary collection as well as ensuring the individual concepts (terms/sensors) are useful and reflective of actually different sensor models. This is especially true of vocabularies as they develop over time, and as corporate memory evolves or is lost.

@rejectedbanana are you a good person from @Sea-BirdScientific to comment on this regarding Sea-Bird sensors?
@vpaba vpaba added the avtt Argo Vocabulary Task Team label Jun 3, 2021
@rejectedbanana
Copy link

@matdon17 I can provide some context with respect to the many version of Sea-Bird CTDs.

The three largest changes to the Sea-Bird Argo can be summarized with the SBE41, SBE41CP and the SBE41N, which reflects changes to sampling as float performance and lifetimes improved. The increasing version numbers reflect minor changes to the firmware, where the sensor hardware stayed the same. The only other change that might need to be noted was the update to the CPU. The old CPU (Alace) was obsolete and redesigned (AlacePlus). The change with regards to the firmware version is noted below.

Alace with obsolete CPU

  • SBE41 and SBE41CP with Firmware 3.x and below

AlacePlus with redesigned CPU

  • SBE41 and SBE41CP with Firmware 7.x.x and above
  • SBE41N with Firmware 5.x.x and above

The IDO and SBE 61 versions roughly follow the same progression in regards to firmware and hardware changes.

In terms of evaluating fleet performance, this may be redundant with the additional metadata on sensors captured by the DAC. But I defer to @apswong and @mbelbeoch in that regard.

@apswong
Copy link

apswong commented Jul 7, 2021

@rejectedbanana, thank you for providing that information - it is very useful. I have a question ... if I have the serial number of the CTD, will I then be able to somehow obtain the firmware version?

@matdon17 @mbelbeoch - I'd like to mention that the version numbers of the sensors have always been available to us. However, some Argo groups choose to include them in the SENSOR_MODEL field in Ref Table 27, and some don't. In that regard, the usage of SENSOR_MODEL is already not uniform across DACs. So even if there is any diagnostic value in including the version number of the sensor, that value is lost because not all DACs record it, and I doubt that DACs will agree to dig into their records and re-populate that field in the meta files.

If information about the firmware revision can be obtained via the serial number of the sensor, which is consistently recorded in SENSOR_SERIAL_NO, then there really is no need to keep the revision numbers in SENSOR_MODEL.

In summary, I would support a motion to clean up SENSOR_MODEL in Ref Table 27. At the very least, we need to stop the proliferation of the same entries that only differ by revision numbers.

@mbelbeoch
Copy link

@apswong
I support it as well !
Mathieu

@apswong
Copy link

apswong commented Jun 15, 2022

We now have a genuine need to record explicitly the sensor firmware version in the Argo meta files. The onboard processing of the RBRargo3 CTD is evolving to include thermal lag correction. The details of this onboard implementation are needed in delayed-mode, and can be obtained if the CTD firmware version is known.

Because of the above reason, and because of the need to clean up SENSOR_MODEL in R27, I suggest the following:

  1. Consolidate the entries in R27 by removing all the firmware versions that are appended to the original sensor model names. For example, instead of having 23 entries of SBE41CP_xxx, there should only be 1 entry, just SBE41CP. The consolidated SENSOR_MODEL will be a constrained vocabulary that adhere to the manufacturer's vocabulary.
  2. Create a new Argo meta file variable SENSOR_FIRMWARE_VERSION. Similar to the other float meta variable FIRMWARE_VERSION, this will be an unconstrained variable.

Name:
SENSOR_FIRMWARE_VERSION
Definition:
char SENSOR_FIRMWARE_VERSION (N_SENSOR, STRING16)
:long_name = "Firmware version of the sensor"
:_FillValue = " "
Comment:
Firmware version of the sensor. Example: 1.141.

How do we make this happen?

@mbelbeoch
Copy link

mbelbeoch commented Jun 15, 2022 via email

@matdon17
Copy link
Author

As I've left NOC/BODC/Argo it's no longer my call, but I would make the following observations:

  • moving to float firmware versions being unconstrained will be a barrier to tracking sensor development and work counter to the objective of understanding how firmware versions might effect data collection - this is a problem with other such unconstrained metadata fields leaving a large and complex legacy to untangle
  • simplifying R27 will greatly help in preventing the rush to populate the relevant metadata when floats are first notified/launched - which is I think part of the reason firmware versions crept in and propagated to begin with
  • a second vocab and Argo meta-data field, specifying firmware version as well might be desirable, although this could be optional rather than compulsory

Now unfollowing this thread.

@tlmaurer
Copy link
Collaborator

@apswong This makes sense to me and is something I would love to see agreement/decision on prior to embarking upon expansion of vocabularies for the pH sensor. There are currently two new pH sensor versions under development that will likely continue to evolve in terms of version # in the near-term. My question at this stage re the CTD - would this be a change moving forward only (how would all the previous meta data for CTD SENSOR_MODEL listed with version number be handled? Would it require reprocessing to retroactively label each CTD with the new scheme?)

@tlmaurer tlmaurer mentioned this issue Dec 9, 2022
Closed
@tcarval tcarval mentioned this issue Dec 9, 2022
Open
@vpaba vpaba transferred this issue from nvs-vocabs/R27 Feb 9, 2023
@vpaba
Copy link
Contributor

vpaba commented Jun 22, 2023

Following the latest @nvs-vocabs/avtt meeting held on the 30th May 2023, @BrianKingNOC has offered to create a task group in charge of finding solutions to record sensor firmware version in the Argo data system, thus to carry this thread fowrard. Please contact Brian for any further comments or ideas!

@vpaba vpaba removed their assignment Jun 22, 2023
@mscanderbeg mscanderbeg added the admt approval requested This ticket is waiting for ADMT approval label Oct 6, 2023
@BrianKingNOC
Copy link

BrianKingNOC commented Oct 9, 2023 •
edited
Loading

I'm not sure where the latest version of what is proposed resides. In the work that I have been doing with RBR and SBS, we have been expecting that SENSOR_MODEL will indeed be simplified, and SENSOR_FIRMWARE_VERSION will become a new variable. We expect that SENSOR_MAKERs will provide the firmware version string to the user. So although the content of SENSOR_FIRMWARE_VERSION will be formally unconstrained, we expect users would simply transcribe the string provided by the sensor maker, so the list of possible values for that string will not expand arbitrarily, and will be meaningful in terms of genuine changes made by the sensor maker, and can be machine audited.

Compared with the proposal by @apswong on 2022 June 15 , I suggest that the length be 32 instead of 16.

@tcarval
Copy link
Contributor

tcarval commented Oct 26, 2023 •
edited
Loading

Here is the addition of SENSOR_FIRMWARE_VERSION in Argo user's manual, §2.4.7.1 "Float sensor information"

SENSOR_FIRMWARE_VERSION char SENSOR_FIRMWARE_VERSION (N_SENSOR, STRING32)SENSOR_FIRMWARE_VERSION :long_name = "Firmware version of the sensor";SENSOR_FIRMWARE_VERSION :_FillValue = " "; Firmware version of the sensor.Example: 1.141

@tcarval tcarval added admt approved and removed avtt Argo Vocabulary Task Team admt approval requested This ticket is waiting for ADMT approval labels Oct 26, 2023
@vpaba vpaba added the R27 label May 30, 2024
@cgourcuf cgourcuf moved this to AVTT approval in AVTT issues management Oct 9, 2024
@tcarval tcarval added the priority priority topic label Oct 23, 2024
@delphinedobler delphinedobler added the documentation Improvements or additions to documentation label Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@apswong apswong

Labels
admt approved documentation Improvements or additions to documentation priority priority topic R27
Projects
AVTT issues management
Status: AVTT approval
Milestone
No milestone
Development

No branches or pull requests
10 participants
@rejectedbanana @tcarval @tlmaurer @matdon17 @mscanderbeg @vpaba @apswong @delphinedobler @mbelbeoch @BrianKingNOC