Automation data

You can connect data from a Process Mining app with data from automations run in the UiPath Platform. This gives you an end-to-end view of the processes that are monitored through UiPath.

When you connect automation data to your process data, you gain a more detailed perspective of the end-to-end process. This provides greater insight into 'human-in-the-loop' operations occurring within the process. Moreover, it enables you to better understand delays between automated and manual steps, as well as between system steps, and also track specific targets.

The object information (object types and object IDs) that is present in both the Process Mining project and in the automation data is used to connect the automation data to your process data.

For example, you can join Coupa system data for the Purchase-to-Pay process with automation data for the invoice processing process, which is a subprocess of the Purchase-to-Pay process, using object type Sales Order object ID SalesOrderID as connecting IDs across data sources.

Prerequisite

It is assumed that you have automation data that is leveraging the Process Tracking Service and has one or more business objects associated with tasks in the automations. Refer to Process Tracking for more information.

Adding automation data

Follow these steps to add automation data to the Input data.

1. Select the Add data icon next to Automation data in the Input data section of the Data transformations editor. The Select automated business process window opens, showing available process automations.
   note

   You can also select Add data from the Manage automation data window to open the Select automated business process window.

2. Select the process or processes that you want to connect to your Process Mining app data.
3. Select Configure.

The automation data is uploaded and the following tables are added to the Automation data list in the Input data section:

• Automation_events
• Queue_items
• Action_center_tasks
• Automation_due_dates

Deleting automation data

Follow these steps to delete automation data.

1. Go to the Select automated business process window.
2. Unselect the process or processes for which you no longer want to load automation data.
3. Select Configure.

Managing automation data

Select the Settings icon to open the Manage automation data window.

note

Manage automation data is not available for Process Optimization apps created from Maestro.

Excluding tables from the automation

By default, the Automation_events, Queue_items, Action_center_tasks, and Automation_due_dates tables are included in the input data for automations.

You can select the tables you want to exclude from the input.

Setting a date range for the automation data

If you want to set a limited or specific timeframe for the automation, you can specify the start and end date that define the timeframe. The specified timeframe applies to all the tables that are part of the input data.

note

This could still include data from outside of the timeframe if a trace continues (or ends) past the chosen end date. This is because the timeframe restriction applies to the start of the trace, not the duration or end of it. Therefore, any data generated by the trace after the chosen end date will still be included in the results.

Refreshing the data

Automation data is refreshed automatically when data is loaded for the processs app.

You can also refresh the data from the Manage automation data window. Follow these steps.

1. Select Refresh data. A confirmation message is displayed.
2. Select Reload to reload all available automation data.

Automation_events table

The Automation_events table stores the automation events and the business objects that are involved.

The following table describes the fields of the Automation_events table.

Field	Type	Description
Trace_ID	text	The unique identifier of the trace.
Process	text	Name of the process. This is selected by the user when connecting automation data. One or multiple traces can be selected.
Event_ID	text	The unique identifier of the event.
Parent_event_ID	text	The Event_ID of the parent event.
Task	text	Name of the automation event.
Object_ID	text	The business object identifier that related to the automation event. This is explicitly set by the user in the automation workflow.
Object_type	text	What type of business object. For example, sales order, invoice, or customer.
Object_interaction	text	Information about the object related to the event. For example, a create interaction or an approve.
Object_properties	text	A JSON value storing all properties related to the object (key-value pairs).
Automation_name	text	Job property from the job that logs the automation event. In case of a “wait for job” event it is the property of the job that is being waited for.
Automation_type	text	The type of the task the robot will execute.
Job_source	text	An indication of where the job was initiated from.
Job_info	text	A short description of the job.
Automation_version	text	The version of the automation.
Host_machine_name	text	The name of the computer or server on which the job is executed.
Robot_name	text	The name of the robot that executed the job.
Robot_type	text	The type of the robot that is responsible for executing the job. For example, "Attended", "Unattended", or "non-production".
Job_ID	text	The automation event identifier. An event may occur multiple times when multiple objects relate to the event.
Queue_item_ID	text	The identifier of the queue item when a queue item relates to the automation event.
Action_center_task_ID	text	The identifier of the task when a task relates to the automation event.
Event_start	timestamp	Timestamp when an automation event started. This is always available.
Event_end	timestamp	Timestamp when an automation event ended. This is only available when the event is finished.

Queue_items table

The Queue_items table stores the queue items that are related to automation events.

The following table describes the fields of the Queue_items table.

Field	Type	Description
Queue_item_ID	text	The identifier to link the queue item to an automation event.
Queue_name	text	The name of the queue the queue item belongs to.
Defer_date	timestamp	The date and time the queue item becomes available for processing.
Due_date	timestamp	The date and time the queue item should be completed.
Creation_timestamp	timestamp	The date and time the queue item was added to the queue.
Start_processing	timestamp	The date and time when the queue item began to be processed.
End_processing	timestamp	The date and time when the queue item was completed.
Processing_time	duration/integer	The number of milliseconds between Start_processing and End_processing .
Processing_exception_reason	text	An indication why the queue item could not be completed or processed successfully.
Progress	text	An indication of the progress of the queue item.
Reference	text	A unique identifier for the queue item.
Retry_number	integer	The number of times the queue item will be attempted again if it fails initially.
Reviewer	text	The user who is responsible for verifying the successful completion of the queue item.
Review_status	text	The status of the queue item indicated by the review.
Robot_name	text	The name of the robot that processed the queue item.
Priority	text	The priority of the item in the queue.
Processing status	text	The status of the queue item while it is in the process of being handled.
Processing_exception_type	text	An indication of whether an error occurred while processing the item.
Robot_type	text	The type of the robot that is responsible for processing the queue item. For example, "Attended", "Unattended", or "non-production".
Is_open	boolean	A derived property from the processing status. The queue item is considered open when the processing status is “new” or “in progress”.

Action_center_tasks table

The Action_center_tasks table stores the tasks from Action Center that are related to automation events.

The following table describes the fields of the Action_center_tasks table.

Field	Type	Description
Action_center_task_ID	text	Identifier to link the task to an automation event.
Title	text	The description of the task.
Catalog	text	The list of tasks that need to be carried out to complete the workflow.
Assignee	text	The user or system responsible for completing the task.
Action	text	The activity or set of activities that need to be performed in order to complete that task.
Priority	text	The priority of the task in the workflow.
Status	text	The actual status of the task in the workflow.
Type	text	A classification of the task.
Creation_timestamp	timestamp	The date and time when the task was created.
Last_assigned_timestamp	timestamp	The date and time when the task was last assigned.
Completion_timestamp	timestamp	The date and time when the task was completed.
Is_completed	boolean	An indication whether the task is completed.
Assigning_time	integer	A derived field based on the difference between creation and last assigned timestamp.
Completion_time	integer	A derived field based on the difference between creation and completion timestamp.
Number_of_users	integer	A derived field based on the number of users that were involved looking at the task events.

Automation_due_dates table

The Automation_due_dates table stores the due dates from queue items and tasks.

The following table describes the fields of the Automation_due_dates table.

Field	Type	Description
Queue_item_ID	text	Only contains values when the due date relates to a queue item.
Action_center_task_ID	text	Only contains values when the due date relates to a task.
Due_date	text	The name of the Due date: <queue name>, <task type> task assignment, or <task type> task completion.
Due_date_status	text	An indication of the status of the task determined based on when it is expected to be completed.
Action_on_overdue	text	Action to take when the task is overdue. Only available on task-related due dates.
Actual_timestamp	timestamp	The date and time the queue item or task was completed.
Expected_timestamp	timestamp	The date and time the queue item or task should be completed.
On_time	boolean	An indication whether the due date is met or not.
Time_difference	integer	The difference between actual and expected time.

Using automation data in transformations

To use the automation data in your transformations, first add the tables to the sources.yml file. Then, you can refer to these source tables in the dbt project like any other input table.

Referencing the source tables in the dbt project

You can use the following code to reference the tables in your dbt project.

with Automation_events as (
    select * from {{ source('sources', 'Automation_events') }}
),

Queue_items as (
    select * from {{ source('sources', 'Queue_items') }}
),

Action_center_tasks as (
    select * from {{ source('sources', 'Action_center_tasks') }}
),

Automation_due_dates as (
    select * from {{ source('sources', 'Automation_due_dates') }}
),

Use cases

Scenario 1. Adding events from automations to the event log

You can add events from an automation to the event log if your Process Mining transformations cover multiple business objects. In this scenario you connect automation data to a process app to it to gain more understanding in specific parts of the process.

note

The example shows how to connect automations to a Sales order object. You can use a different object by adjusting the relevant references as needed.

Follow these steps to add automations to the event log.

1. Add a new SQL file Sales_order_automation_events.
2. Copy the following SQL example in which you:
   1. Filter the Automation_events on the sales order object type.
   2. Rename Object_ID to Sales_order_ID to identify this is a sales order event.
   3. Use the value stored in Task as the Activity.
   4. Include at least the mandatory field Event_end.

      with Automation_events as (
          select * from {{ source('sources', 'Automation_events') }}
      ),
      
      Sales_order_automation_events as (
          select
              Automation_events."Object_ID" as "Sales_order_ID",
              Automation_events."Task" as "Activity",
              Automation_events."Event_end",
              Automation_events."Event_ID",
              'null' as "Parent_event_ID"
          from Automation_events
          where Automation_events."Object_type" = 'Sales order'
      )
      
      select * from Sales_order_automation_events

3. Union the Sales_order_automation_events with the other events defined in your Process Mining project.
   note

   When you use the Custom or Event log app template, your Process Mining project only tracks one object of interest. You can union the automation events on your object directly with the event log.

   • Make sure to generate a unique Event_ID on the unioned events to ensure unique event identifiers in your event log.
4. Verify that the object ID you defined in the Process Mining project matches the object ID from your automation events. Update your transformations accordingly such that events from both data sources will be connected with the correct objects.

Scenario 2: Getting additional object information from automations

In this scenario you extract the object properties information from the automation data to use it to enrich the dashboards in your process app.

The field Object_properties stores the properties of the business objects that are added in the automations. The value in this field is a JSON format with key-value pairs.

For example: {"Claim_number": 216, "Client": "Alex Smith"}

Extract the individual properties by using the pm-utils json() function.

The following SQL code shows an example of how to use the pm-utils json() function to extract the Claim_number and Client properties.

select
    Automation_events."Object_ID",
    {{ pm_utils.json('Automation_events."Object_properties"', 'Claim_number') }} as "Claim_number",
    {{ pm_utils.json('Automation_events."Object_properties"', 'Client') }} as "Client",
from Automation_events

Scenario 3: Enriching automation events with queue item data

An automation event can be associated with a queue item. The automation event represents the execution of the queue item. The information about queue items is stored in the separate table Queue_items. Join the automation events with this table on the Queue_item_ID to get information about:

• The priority of the execution.
• Exceptions.
• Due dates.

The following SQL code shows an example of how to enrich the automation events with the queue item properties Priority and Processing_exception_type.

select
    Automation_events."Event_ID",
    Automation_events."Object_ID",
    Queue_items."Priority",
    Queue_items."Processing_exception_type"
from Automation_events
left join Queue_items
    on Automation_events."Queue_item_ID" = Queue_items."Queue_item_ID"

Due date information is stored in the Automation_due_dates table. One queue item can have at most one due date. The following SQL code shows how to get the due date information available.

select
    Automation_events."Event_ID",
    Automation_events."Object_ID",
    Queue_items."Due_date",
    Queue_items."Expected_timestamp",
    Queue_items."Actual_timestamp"
from Automation_events
left join Automation_due_dates
    on Automation_events."Queue_item_ID" = Automation_due_dates."Queue_item_ID"

Scenario 4: Enriching automation events with task data

An automation event can be associated with a task. The automation event represents the step in the automated process where a human is involved. The information about tasks is stored in the separate table Tasks. Join the automation events with this table on the Task_ID to get information about:

• The assignee of the task
• The priority of the task
• Due dates (task SLAs)

The following SQL code shows an example on how to enrich the automation events with the task properties Assignee and Priority.

select
    Automation_events."Event_ID",
    Automation_events."Object_ID",
    Action_center_tasks."Assignee",
    Action_center_tasks."Priority"
from Automation_events
left join Action_center_tasks
    on Automation_events."Event_ID" = Action_center_tasks."Action_center_tasks_ID"

Due date information is stored in the Automation_due_dates table. A task can be related to multiple due dates. You can apply due dates when the task should be assigned and when the task should be completed.

note

Joining the Automation_events with Action_center_tasks on the Action_center_task_ID without any filtering may cause duplication.

The following SQL code shows an example of how to enrich the automation events with task due date information by only considering the task completion due dates.

with Task_completion_due_dates as (
    select * from Automation_due_dates
    where pm_utils.charindex('task completion', '"Due_date"') > 0
)

select
    Automation_events."Action_center_tasks_ID",
    Automation_events."Object_ID",
    Action_center_tasks."Due_date",
    Action_center_tasks."Expected_timestamp",
    Action_center_tasks."Actual_timestamp"
from Automation_events
left join Task_completion_due_dates
    on Automation_events."Task_ID" = Task_completion_due_dates."Task_ID"