Data table

The data table structure is the smallest unit of the data API. Its purpose is to organize data of any supported kind into an annotated table and store all required metadata to recover the data from the file system.

A data table is located in a directory within the file system and contains following parts:

The numeric folders are also referred as “Row folders”. Please visit the row folder documentation for more info.

Here you see an example of such a folder that stores three data rows. Here we assume that the stored data types are imagej-imgplus (ImageJ images).

graph LR; Root["/"] --> DataTable["data-table.json"]; Root --> R0["0/"]; Root --> R1["1/"]; Root --> R2["2/"]; R0 --> R0Img["data.tif"]; R1 --> R1Img["data.tif"]; R2 --> R2Img["data.tif"];

data-table.json

Here you will find the structure of the data-table.json file. You also can access the document in JSON Schema format: data-table.schema.json. Please note that the description for a data table row (data-table-row.schema.json) can be found below.

Here is an example data table that stores FFT images:

{
  "node-id" : "ij1-fft-forward2d",
  "slot" : "Output",
  "internal-path" : "/fastdata/projects/JIPipe/Output",
  "rows" : [ {
    "index" : 0,
    "annotations" : [ ],
    "data-annotations": [ ],
    "true-data-type" : "imagej-imgplus-fft-2d"
  } ],
  "data-type" : "imagej-imgplus-fft-2d"
}

Data table

Defines a data table row

node-id: string

The node type ID that generated the data. Optional.

slot: string

The slot name that generated the data. Optional.

rows: object[]

The data row items

data-type: string

The data type ID behind the row

Data table row

Defines a data table row

id: number

Row index within the data table

annotations: object[]

Annotations attached to the data row

data-annotations: object[]

Data annotations attached to the data row

true-data-type: string

The true (not the slot) data type ID behind the row

Annotation

An annotation

name: string

Name of the annotation

value: string

Value of the annotation

Data annotation

A data annotation

name: string

Name of the annotation

true-data-type: string

Data type ID of the stored data annotation

row-storage-folder: string

Relative path to the current data table containing the data annotation data row