Skip to content

2.3 ChRIS DS plugin workflow: extract DICOM metadata.

Jump to bottom
Rudolph Pienaar edited this page Sep 21, 2018 · 6 revisions

2.3 ChRIS DS plugin workflow: extract DICOM metadata.

Abstract

This page describes interacting with the ChRIS Ultron Back End (CUBE) using pfurl and running a DS type plugin called pl-pfdicom_tagextract. This plugin wraps around an underlying pfdicom_tagExtract python module and provides the ability to extract DICOM meta data values and generate reports in a variety of formats.

Preconditions

An existing FS feed containing the DICOM data to anomymize.

Please follow the steps here to create a new source, or FS, feed containing data to anonymize.

Cleared internal pman database

Be sure that the internal pman job database has been cleared after instantiating CUBE and running the test suite. If this is not done, then collisions in the job-ID jid space will occur -- some of the tests will have the same jid as actual feeds and operations will fail.

pfurl --verb POST --raw --http ${HOST_IP}:5010/api/v1/cmd \
      --jsonwrapper 'payload' --msg \
 '{  "action": "DBctl",
         "meta": {
                 "do":     "clear"
         }
 }' --quiet --jsonpprintindent 4

Set some convenience environment variables

In a new terminal, set HOST_IP to the IP of the actual host you are using. In Linux, this is

export HOST_IP=$(ip route | grep -v docker | awk '{if(NF==11) print $9}')
export HOST_PORT=8000

Backend user

At least one user with name/password combination as

  • chris/chris1234

The FS example will take care of this.

Containerized data/processing services:

  • pfcon
  • pfioh
  • pman

The FS example will take care of this.

GET list of plugins and check for pl-pfdicom_tagextract plugin:

Call

pfurl --auth chris:chris1234                         \
      --http ${HOST_IP}:${HOST_PORT}/api/v1/plugins/ \
      --quiet --jsonpprintindent 4

Look through the list for pl-pfdicom_tagextract. For example, let's say that this is located at

http://10.17.24.163:8000/api/v1/plugins/8/

we can call

export P=8
pfurl --auth chris:chris1234                        \
      --http ${HOST_IP}:8000/api/v1/plugins/$P/     \
      --quiet --jsonpprintindent 4

Receive

{
  "collection": {
    "version": "1.0",
    "href": "http://10.72.76.155:8000/api/v1/plugins/8/",
    "items": [
      {
        "data": [
          { "name": "name", "value": "pfdicom_tagextract" },
          { "name": "dock_image", "value": "fnndsc/pl-pfdicom_tagextract" },
          { "name": "type", "value": "ds" },
          { "name": "authors", "value": "FNNDSC ([email protected])" },
          {
            "name": "title",
            "value": "A DICOM tag extractor/reporting tool. Generates reports based on DICOM header information."
          },
          { "name": "category", "value": "DICOM" },
          {
            "name": "description",
            "value": "This app performs a recursive walk down an input tree, and for each location with a DICOM file, will generate a report in the corresponding location in the output tree."
          },
          {
            "name": "documentation",
            "value": "https://github.com/FNNDSC/pfdicom_tagExtract"
          },
          { "name": "license", "value": "Opensource (MIT)" },
          { "name": "version", "value": "1.0" },
          { "name": "execshell", "value": "python3" },
          { "name": "selfpath", "value": "/usr/src/dcm_tagExtract" },
          { "name": "selfexec", "value": "dcm_tagExtract.py" },
          { "name": "compute_resource_identifier", "value": "host" },
          { "name": "min_number_of_workers", "value": 1 },
          { "name": "max_number_of_workers", "value": 1 },
          { "name": "min_cpu_limit", "value": 1000 },
          { "name": "max_cpu_limit", "value": 2147483647 },
          { "name": "min_memory_limit", "value": 200 },
          { "name": "max_memory_limit", "value": 2147483647 },
          { "name": "min_gpu_limit", "value": 0 },
          { "name": "max_gpu_limit", "value": 0 }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/8/",
        "links": [
          {
            "rel": "parameters",
            "href": "http://10.72.76.155:8000/api/v1/plugins/8/parameters/"
          },
          {
            "rel": "instances",
            "href": "http://10.72.76.155:8000/api/v1/plugins/8/instances/"
          }
        ]
      }
    ],
    "links": [ ]
  }
}

GET parameter info for plugin

Follow the links provided to get information on the parameters for the plugin

Call

pfurl --auth chris:chris1234     \
      --http ${HOST_IP}:${HOST_PORT}/api/v1/plugins/$P/parameters/ \
      --quiet --jsonpprintindent 4

Receive

{
  "collection": {
    "version": "1.0",
    "href": "http://10.72.76.155:8000/api/v1/plugins/8/parameters/",
    "items": [
      {
        "data": [
          { "name": "name", "value": "inputFile" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "" },
          { "name": "flag", "value": "-i" },
          { "name": "action", "value": "store" },
          { "name": "help", "value": "input file" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/43/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "extension" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "" },
          { "name": "flag", "value": "-e" },
          { "name": "action", "value": "store" },
          { "name": "help", "value": "DICOM file extension" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/44/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "tagFile" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "" },
          { "name": "flag", "value": "-F" },
          { "name": "action", "value": "store" },
          { "name": "help", "value": "file containing tags to parse" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/45/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "tagList" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "" },
          { "name": "flag", "value": "-T" },
          { "name": "action", "value": "store" },
          { "name": "help", "value": "comma-separated tag list" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/46/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "rawType" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "raw" },
          { "name": "flag", "value": "-r" },
          { "name": "action", "value": "store" },
          { "name": "help", "value": "display raw tags" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/47/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "imageFile" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "" },
          { "name": "flag", "value": "-m" },
          { "name": "action", "value": "store" },
          { "name": "help", "value": "image file to convert DICOM input" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/48/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "imageScale" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "" },
          { "name": "flag", "value": "-s" },
          { "name": "action", "value": "store" },
          {
            "name": "help",
            "value": "scale images with factor and optional :interpolation"
          }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/49/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "outputFileStem" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": false },
          { "name": "default", "value": "" },
          { "name": "flag", "value": "-o" },
          { "name": "action", "value": "store" },
          { "name": "help", "value": "output file" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/50/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "outputFileType" },
          { "name": "type", "value": "string" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "raw" },
          { "name": "flag", "value": "-t" },
          { "name": "action", "value": "store" },
          { "name": "help", "value": "list of output report types" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/51/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      },
      {
        "data": [
          { "name": "name", "value": "printElapsedTime" },
          { "name": "type", "value": "boolean" },
          { "name": "optional", "value": true },
          { "name": "default", "value": "False" },
          { "name": "flag", "value": "--printElapsedTime" },
          { "name": "action", "value": "store_true" },
          { "name": "help", "value": "print program run time" }
        ],
        "href": "http://10.72.76.155:8000/api/v1/plugins/parameters/52/",
        "links": [{ "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" }]
      }
    ],
    "links": [
      { "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" },
      {
        "rel": "next",
        "href": "http://10.72.76.155:8000/api/v1/plugins/8/parameters/?limit=10&offset=10"
      }
    ]
  }
}

Run an instance of the plugin

Call

Using httpie

http -a chris:chris1234 POST http://${HOST_IP}:${HOST_PORT}/api/v1/plugins/$P/instances/ \
Content-Type:application/vnd.collection+json \
Accept:application/vnd.collection+json \
template:='{"data":
    [
     {"name":"outputFileType",
      "value":"jpg"},
     {"name":"sliceToConvert",
      "value":"-1"},
     {"name":"func",
      "value":"invertIntensities"},
     {"name":"previous_id",
      "value":"1"}
    ]
}'

Using pfurl

pfurl --auth chris:chris1234 --verb POST --http ${HOST_IP}:${HOST_PORT}/api/v1/plugins/$P/instances/ \
--content-type application/vnd.collection+json \
--jsonwrapper 'template' --msg '
{"data":
    [
     {"name":"extension",
      "value":"dcm"},
     {"name":"threads",
      "value":"0"},
     {"name":"imageScale",
      "value":"2"},
     {"name":"outputFileStem",
      "value":"%PatientID-%PatientAge"},
     {"name":"outputFileType",
      "value":"raw,json,html,dict,col,csv"},
     {"name":"imageFile",
      "value":"m:%_nospc|-_ProtocolName.jpg"},
     {"name":"useIndexhtml",
      "value":"True"},
     {"name":"verbosity",
      "value":"5"},
     {"name":"previous_id",
      "value":"1"}
    ]
}' \
--quiet --jsonpprintindent 4

Receive

{
  "stdout": {
    "collection": {
      "version": "1.0",
      "href": "http://10.72.76.155:8000/api/v1/plugins/8/instances/",
      "items": [
        {
          "data": [
            { "name": "id", "value": 2 },
            { "name": "previous_id", "value": 1 },
            { "name": "plugin_name", "value": "pfdicom_tagextract" },
            { "name": "start_date", "value": "2018-09-17T12:46:02.738334-04:00" },
            { "name": "end_date", "value": "2018-09-17T12:46:02.738375-04:00" },
            { "name": "status", "value": "started" },
            { "name": "owner", "value": "chris" },
            { "name": "compute_resource_identifier", "value": "host" },
            { "name": "cpu_limit", "value": 1000 },
            { "name": "memory_limit", "value": 200 },
            { "name": "number_of_workers", "value": 1 },
            { "name": "gpu_limit", "value": 0 }
          ],
          "href": "http://10.72.76.155:8000/api/v1/plugins/instances/2/",
          "links": [
            {
              "rel": "previous",
              "href": "http://10.72.76.155:8000/api/v1/plugins/instances/1/"
            },
            { "rel": "plugin", "href": "http://10.72.76.155:8000/api/v1/plugins/8/" },
            {
              "rel": "string_param",
              "href": "http://10.72.76.155:8000/api/v1/plugins/string-parameter/1/"
            },
            {
              "rel": "string_param",
              "href": "http://10.72.76.155:8000/api/v1/plugins/string-parameter/2/"
            },
            {
              "rel": "string_param",
              "href": "http://10.72.76.155:8000/api/v1/plugins/string-parameter/3/"
            },
            {
              "rel": "string_param",
              "href": "http://10.72.76.155:8000/api/v1/plugins/string-parameter/4/"
            },
            {
              "rel": "string_param",
              "href": "http://10.72.76.155:8000/api/v1/plugins/string-parameter/5/"
            },
            {
              "rel": "string_param",
              "href": "http://10.72.76.155:8000/api/v1/plugins/string-parameter/6/"
            },
            {
              "rel": "string_param",
              "href": "http://10.72.76.155:8000/api/v1/plugins/string-parameter/7/"
            },
            {
              "rel": "bool_param",
              "href": "http://10.72.76.155:8000/api/v1/plugins/bool-parameter/1/"
            }
          ]
        }
      ],
      "links": [ ]
    }
  },
  "msg": "push OK."
}

Query job state (and also trigger a file registration in CUBE)

Examine the return JSON for the instance ID returned. In the above, we note that this is

{
  "stdout": {
    "collection": {
      "version": "1.0",
      "href": "http://10.72.76.155:8000/api/v1/plugins/8/instances/",
      "items": [
        {
          "data": [
            { "name": "id", "value": 2 },
            { "name": "previous_id", "value": 1 },
            { "name": "plugin_name", "value": "pfdicom_tagextract" },
            { "name": "start_date", "value": "2018-09-17T12:46:02.738334-04:00" },
            { "name": "end_date", "value": "2018-09-17T12:46:02.738375-04:00" },
            { "name": "status", "value": "started" },
            { "name": "owner", "value": "chris" },
            { "name": "compute_resource_identifier", "value": "host" },
            { "name": "cpu_limit", "value": 1000 },
            { "name": "memory_limit", "value": 200 },
            { "name": "number_of_workers", "value": 1 },
            { "name": "gpu_limit", "value": 0 }
          ],
          "href": "http://10.72.76.155:8000/api/v1/plugins/instances/2/",

In other words, instance 2.

Call

Using pfurl

pfurl --auth chris:chris1234                               \
      --verb GET                                           \
      --http ${HOST_IP}:${HOST_PORT}/api/v1/plugins/instances/2/   \
      --content-type application/vnd.collection+json       \
      --quiet --jsonpprintindent 4

Receive

{
  "collection": {
    "version": "1.0",
    "href": "http://192.168.1.189:8000/api/v1/plugins/instances/2/",
    "items": [
      {
        "data": [
          { "name": "id", "value": 2 },
          { "name": "previous_id", "value": 1 },
          { "name": "plugin_name", "value": "pfdicom_tagextract" },
          { "name": "start_date", "value": "2018-09-21T18:26:33.273411-04:00" },
          { "name": "end_date", "value": "2018-09-21T18:27:23.556806-04:00" },
          { "name": "status", "value": "finishedSuccessfully" },

where we note that status is finishedSuccessfully.

-30-

Clone this wiki locally