Custom templates¶

Offcourse you can create your own templates. Basically a template is a yaml file and some jinja templating (https://jinja.palletsprojects.com/) magic to make it configurable. There are two main levels defined for each template.

pipeline
scripts

We will make use of some yaml and jinja features to not have to define some of the variables multiple times. We added those at the top of the file in a key called basic-settings. All values (and keys) at the pipeline level are available when rendering the pipeline config.xml. All key, values at the scripts level are available when rendering script files. Values that need to be defined at both levels could be added at the beginning of the template and referenced to from both pipeline and scripts levels.

Creating a template¶

To do so we first create a folder ie: templates in the root of our project folder. In this folder we create a new yaml file ie: my_template.yaml. First things to define are:

Version of the template defition file, this should be 1.
Location where CTP can store data like quarantine and root folders.
Define server properties like maxThreads and port number for the web interface.

This should look like this:

basic-settings:
  quarantine: &quarantine {{server.storage}}/quarantine/{{pipeline.name}}
  root: &root {{server.storage}}/root/{{pipeline.name}}
version: 1
pipeline:
  name: {{pipeline.name}}
  version: 1.0
  author: CTP Admin
  server:
    maxthreads: 20
    port: {{server.port}}

After defining the basic settings we can start defining the stages, and script properties that are part of the pipeline. To keep the pipeline flexible we probably want to make the import and export stages flexible like in the default templates. These should look like this:

pipeline:
  stages:
    import:
      type: {{pipeline.import_type}}Import
      {% if ((pipeline.import_type == 'Dicom') or (pipeline.import_type == 'Http')) %}
      port: {{pipeline.import_port}}
      {%endif%}
      {% if ((pipeline.import_type == 'Directory') or (pipeline.import_type == 'Archive')) %}
      path: {{pipeline.import_path}}
      {%endif%}
      quarantine: *quarantine
      root: *root
    export:
      type: {{pipeline.export_type}}Export
      {% if ((pipeline.export_type == 'Dicom') or (pipeline.export_type == 'Http')) %}
      url: {{pipeline.export_url}}
      {% elif ((pipeline.export_type == 'File') or (pipeline.export_type == 'Directory')) %}
      path: {{pipeline.export_path}}
      {%endif%}
      quarantine: *quarantine
      root: *root

This gives the option to define an import and export stage with the following properties:

type: The type of import or export stage.
port: The port number the import stage should listen to. (Only for Dicom and Http)
path: The path where the import or export stage should read or write data to. (Only for Directory and Archive)
url: The url where the export stage should send data to. (Only for Http)

This already gives a working template that can be used to create a CTP configuration. To add to option to enable/disable tracking of DICOM files we could add the following to the template:

{% if pipeline.object_tracker %}
object_input_tracker:
    type: ObjectTracker
    root: *root
{% endif %}

Adding scripts¶

Some stages requires scripts these can also be templated using CTP builder and just lie the pipeline these can be made configurable using Jinja templating. For example we could add a script to filter out DICOMS with series description we know might contain personal information or are not of interest in this study. So first let’s create a new script file in the scripts folder ie: SeriesFilter.script. The content of this file should look like this:

{%- for series_description in series_descriptions -%}
{%- if loop.first %}
! SeriesDescription.equalsIgnoreCase("{{series_description}}")
{%- else -%}
* !SeriesDescription.equalsIgnoreCase("{{series_description}}")
{%- endif -%}
{%- endfor -%}

This will create a filter for all series_descriptions in the variable series_descriptions. There a couple of acttions we have to do to make sure this end up in the CTP configuration:

Add a stage to the pipeline.
Add the script to make sure it is included in the configuration.
Make sure the series descriptions are configurable.

The final pipeline template should look like:

basic-settings:
  quarantine: &quarantine {{server.storage}}/quarantine/{{pipeline.name}}
  root: &root {{server.storage}}/root/{{pipeline.name}}
version: 1
pipeline:
  name: {{pipeline.name}}
  version: 1.0
  author: CTP Admin
  server:
    maxthreads: 20
    port: {{server.port}}
  stages:
    import:
      type: {{pipeline.import_type}}Import
      {% if ((pipeline.import_type == 'Dicom') or (pipeline.import_type == 'Http')) %}
      port: {{pipeline.import_port}}
      {%endif%}
      {% if ((pipeline.import_type == 'Directory') or (pipeline.import_type == 'Archive')) %}
      path: {{pipeline.import_path}}
      {%endif%}
      quarantine: *quarantine
      root: *root
    {% if pipeline.object_tracker %}
    object_input_tracker:
        type: ObjectTracker
        root: *root
    {% endif %}
    {% if scripts.series_descriptions %}
    series_filter:
      type: Filter
      script: scripts/SeriesFilter.script
      root: *root
      quarantine: *quarantine
    {% endif %}
    export:
      type: {{pipeline.export_type}}Export
      {% if ((pipeline.export_type == 'Dicom') or (pipeline.export_type == 'Http')) %}
      url: {{pipeline.export_url}}
      {% elif ((pipeline.export_type == 'File') or (pipeline.export_type == 'Directory')) %}
      path: {{pipeline.export_path}}
      {%endif%}
      quarantine: *quarantine
      root: *root
{% if scripts.series_descriptions %}
scripts:
  series_descriptions:
    {{ scripts.series_descriptions }}
  include:
    - SeriesFilter.script
{% endif %}

This should finish the new template. Now we need to make a configuration to render the template into a configuration. The configuration looks like:

version: 2
template: my_template
server:
  port: 8080
  storage: /mnt/data
pipeline:
  name: anonymization_project
  import_type: Dicom
  import_port: 1080
  export_type: Dicom
  export_url: https://xnat.example.com:8143
  object_tracker: True
scripts:
  series_descriptions:
    - Report
    - Screenshot
    - Dose
    - Exam