Edge Compute Device

Post-Processing

9min

Overview
With Canopy Vision, when you are running a vision pipeline with model inference enabled, the output data stream from the model can be passed to either or both of the following:
Save raw data to CSV files
Pass raw data to a Post-Processing script
The Post-Processing script is a Python-based script that is written to parse the model output data and apply any necessary business logic. For classificaton models, this is simply the class label and confidence data. For object detection models, this is the bounding box data.
This help article will show how to parse the raw data and write your own post-processing script for an object detection model.
Settings
In order to configure the vision pipeline to output data to your post-prociessing script, the config file will need to be edited with the following settings for the data_sink  section:
data_sink settings
[data_sink]
enable = 1
stream = 1
interval = 5
postprocessing-enable = 1
postprocessing-script = streaming_print.py
save-raw-data = 1
max-dir-size = 2

[data_sink]
enable = 1
stream = 1
interval = 5
postprocessing-enable = 1
postprocessing-script = streaming_print.py
save-raw-data = 1
max-dir-size = 2
﻿
For the data_sink block, the following parameter descriptions are as follows:
Property Name
Values
Description
enable
bool
Enables data output
stream
bool
Sets mode of data output to streaming or batched output
interval
int
Only applies when stream is set to 0. Corresponds to the number of frames to batch before sending data
postprocessing-enable
bool
Determines whether to pass data to a post-processing script
postprocessing-script
string
Path to your custom post-processing script
save-raw-data
bool
Toggles whether to save raw output data (bounding boxes or frame classifications) to CSV files on disk (/mnt/media/model_output)
max-dir-size
int
Maximum size of files in GB for the model_output directory when save-raw-data is enabled above. If this directory exceeds the set value, the oldest files will be deleted to get the directory size back below the set value
Raw Data Format
For object detection models, the raw data that is passed from the model to the post-prociessing script will have the following format for every frame(image):
Example Data - Object Detection
{
   "timestamp":1640611347.163409,
   "data":[
      {
         "class_id":1,
         "label":"person",
         "confidence":0.92418234,
         "top":100.80000305175781,
         "left":396.8000183105469,
         "width":189.1555633544922,
         "height":240.00001525878906
      },
      {
         "class_id":2,
         "label":"bag",
         "confidence":0.52521923,
         "top":100.80000305175781,
         "left":396.8000183105469,
         "width":189.1555633544922,
         "height":240.00001525878906
      }
   ]
}
{
   "timestamp":1640611347.163409,
   "data":[
      {
         "class_id":1,
         "label":"person",
         "confidence":0.92418234,
         "top":100.80000305175781,
         "left":396.8000183105469,
         "width":189.1555633544922,
         "height":240.00001525878906
      },
      {
         "class_id":2,
         "label":"bag",
         "confidence":0.52521923,
         "top":100.80000305175781,
         "left":396.8000183105469,
         "width":189.1555633544922,
         "height":240.00001525878906
      }
   ]
}
﻿
Data Flow
When streaming data to a post-processing script, the script is started by the vision pipeline and data flows to it via standard output stdout. To access this data within the Python process, you will need to loop while waiting for input() from stdout as follows:
Receiving data via stdout
while True:
    try:
        data = input()
    except EOFError:
        continue
while True:
    try:
        data = input()
    except EOFError:
        continue
﻿
Every time there's a frame with at least one object detected, the vision pipeline outputs the raw bounding box data and will be captured by data = input() 
Keep in mind that post-processing logic may slow down the overall performance (FPS) of the vision pipeline if there are slow or blocking operations. For example, if you are making an API POST request to send data to an external endpoint, that network call may take some time to complete, delaying the vision pipeline and lowering the FPS. For these situations, it is often best to use a background thread or other asyncronous methods.
Output, such as print statements and exceptions, can be viewed by observing the logs of the vision pipeline:  journalctl -fu canopy-deepstream 
Example Script
The following script is an example post-processing script that enables a siren while a person is detected within the frame:
Example post-processing script
#!/usr/bin/env python3

import json
from urllib import request
import actuator
from datetime import datetime
import uuid
from collections import deque
from concurrent.futures import ThreadPoolExecutor
from threading import Lock
import atexit

uuid_str = str(uuid.uuid4())[:4]
print(f"{uuid_str}: starting script at {datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]}")

executor = ThreadPoolExecutor(max_workers=2)
siren_lock = Lock()
siren_activated = False
running = True

def teardown():
    global running 
    running = False
    executor.shutdown()
atexit.register(teardown)

def siren():
    while running:
        if siren_activated:
            siren_lock.acquire()
            actuator.siren_run(0.5)
            siren_lock.release()

executor.submit(siren)

while True:
    try:
        data = input()
    except EOFError:
        continue
    data = json.loads(data)
    person_detected = any((obj["label"] == "person") for obj in data["data"])
    if person_detected:
        siren_activated = True
    else:
        siren_activated = False

print("Exiting...")
#!/usr/bin/env python3

import json
from urllib import request
import actuator
from datetime import datetime
import uuid
from collections import deque
from concurrent.futures import ThreadPoolExecutor
from threading import Lock
import atexit

uuid_str = str(uuid.uuid4())[:4]
print(f"{uuid_str}: starting script at {datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]}")

executor = ThreadPoolExecutor(max_workers=2)
siren_lock = Lock()
siren_activated = False
running = True

def teardown():
    global running 
    running = False
    executor.shutdown()
atexit.register(teardown)

def siren():
    while running:
        if siren_activated:
            siren_lock.acquire()
            actuator.siren_run(0.5)
            siren_lock.release()

executor.submit(siren)

while True:
    try:
        data = input()
    except EOFError:
        continue
    data = json.loads(data)
    person_detected = any((obj["label"] == "person") for obj in data["data"])
    if person_detected:
        siren_activated = True
    else:
        siren_activated = False

print("Exiting...")
﻿
Steps for Changing Post-Processing Scripts
The following steps should be taken to enable post-processing with a new custom script:
Make sure the script is copied to the device. Typical locations for this script would be in the /etc/canopy directory or within the specific model sub-folder (ie /var/lib/canopy/models/peoplenet/)
Ensure the script is executable: sudo chmod +X /path/to/postprocessing.py 
Change the settings/configs at /etc/canopy/ds_config.txt to point to this new post-processnig script location
Restart the Canopy Deepstream application sudo systemctl restart canopy-deepstream 
Note: The data_sink output, which includes custom post-processing scripts, only works if model inference is being performed. This can be confirmed by checking the ds_config.txt file to ensure the primary-gie has enable = 1 
﻿

Property Name	Values	Description
enable	bool	Enables data output
stream	bool	Sets mode of data output to streaming or batched output
interval	int	Only applies when stream is set to 0. Corresponds to the number of frames to batch before sending data
postprocessing-enable	bool	Determines whether to pass data to a post-processing script
postprocessing-script	string	Path to your custom post-processing script
save-raw-data	bool	Toggles whether to save raw output data (bounding boxes or frame classifications) to CSV files on disk (/mnt/media/model_output)
max-dir-size	int	Maximum size of files in GB for the model_output directory when save-raw-data is enabled above. If this directory exceeds the set value, the oldest files will be deleted to get the directory size back below the set value

Updating Model Settings

Configuring Camera Input