Reporting and Maintaining Progress in Your Nom Nom App

Reporting and Maintaining Progress in Your Nom Nom App

This article explains the best practices for reporting Task progress in your Nom Nom App code and for updating a parameter value in a Task from your Nom Nom App code so that you can maintain progress across multiple Task executions.  Specifically it focuses on using the engine.update_progress and the engine.update_parameter functions.  It is assumed that the reader is already familiar with the information in the Creating Your First Nom Nom App article.

Reporting Progress During Task Execution

The following examples demonstrate the use of the engine.update_progress function to inform users how far your code has progressed during various points of a Task execution.  The progress variable is passed into the function with a value set between 0 and 100.  Each time the function is called, this portion of the Nominode Task user interface is updated to display the value passed to the function as a percentage and to update the fill amount in a progress bar:



The function could just be called between sections of your App code:

download_data_function()
engine.update_progress(progress=25)
process_data_function()
engine.update_progress(progress=50)
upload_processed_data_function()
engine.update_progress(progress=75)
cleanup_function()
engine.update_progress(progress=100)

But a more useful application is to find the portions of your App code where it is looping several times and call the function at certain intervals within those loops passing in progress values based on the size of the loop:

i = 0
total = 5000
while i < total:
     i += 1
     five_percent = round(total * .05) or 1
     if i % five_percent == 0:
          p = "%.2f" % ((i / total) * 100)
          engine.update_progress(progress=p)
          logger.info(f"Processed {p}% of the data.")

Retaining Progress Between Task Executions

Many Nom Nom Apps are designed to be run on a regular schedule using a Task triggered by a Sequence rather than just being run a single time.  And often they are running to download or process "new" data.  Meaning they want to begin their activity at the point in the data where the previous Task execution ended.  The following example demonstrates the use of the engine.update_parameter function to write the datetime stamp of when the Task began its execution to a parameter called last_run in the Task:

start_time = datetime.strftime(datetime.now(), "%Y-%m-%d %H:%M:%S%z")
download_new_data_function(start_point=params["last_run"])
engine.update_parameter("last_run", start_time)

The value stored and updated in the last_run parameter is also passed as the starting point value into the function that downloads data, which ensures that, each time the Task runs, it is downloading data created between the previous time it ran and the current time.  This example relies on the data to download being sorted by a created or updated date and the API used to access it including an option to allow subset extraction based on a date.  But the function is equally applicable to any type of persistent pointer value that can be used to traverse through a set of data via multiple Task executions.

    • Related Articles

    • Testing Your Nom Nom App

      Local Testing For Nom Nom Apps created in Python, we recommend using pytest to locally test the code that you have written.  Follow this link for a general overview on how to install and use pytest.  In this article, we will discuss a few suggestions ...

    • Creating Your First Nom Nom App

      In this tutorial, we will be examining the contents of the template Nom Nom App generated from the SDK and making it available to Tasks on your Nominode. Prerequisites Python 3.7.4 (or newer) https://www.python.org/downloads/ Docker 19.03 (or newer) ...

    • Adding Your Nom Nom App to Our Store

      When you deploy a Nom Nom App that you have created, it is immediately available to be used on all of the Nominodes within your Nom Nom Data organization.  However, if you want to share your Nom Nom App with others outside of your organization, ...

    • Using an External Process in Your Nom Nom App

      This article explains how to leverage an external process from your Nom Nom App.  Specifically it discusses how to use the Scrapy python package.  It is assumed that the reader is already familiar with the information in the Creating Your First Nom ...

    • Adding a Connection to Your Nom Nom App

      In this tutorial, we will discuss how to add a Connection parameter to the template Nom Nom App generated from the SDK and how to use it to connect to Slack.  More information about creating Connections is available in the Managing Connections on a ...