egse.process

cgse-common

This code is part of the cgse-common package.

egse.process ¶

This module provides functions and classes to work with processes and sub-processes.

We combine two great packages, the subprocess and the psutils packages, to provide a simplified, robust and user-friendly interface to work with sub-processes. The classes and functions are optimized to work with processes within the framework of the cgse, so we do not intend to be fully generic. If you need that, we recommend to use the subprocess and psutil packages directly.

The main class provided is the SubProcess which by default, starts a sub-process in the background and detached from the parent process. That means there is no communication between the parent and the subprocess through pipes. Most (if not all) processes in the cgse framework communicate with ZeroMQ messages in different protocols like REQ-REP, PUSH-PULL and ROUTER-DEALER

Another useful class is the ProcessStatus. This class provides status information like memory and CPU usage for the running process. Additionally, it will generate and update metrics that can be queried by the Prometheus timeseries database.

Classes:

Name	Description
`ProcessStatus`	The ProcessStatus is basically a dataclass that contains the status information of a running
`SubProcess`	A SubProcess that is usually started by the ProcessManager.

Functions:

Name	Description
`get_process_info`	Loops over all running processes and tries to match each item in 'items' to the command line
`is_process_running`	Check if there is any running process that contains the given items in its
`list_processes`	Returns and optionally prints the processes that match the given criteria in items.
`list_zombies`	Returns a list of zombie processes.

ProcessStatus ¶

ProcessStatus(*, metrics_prefix=None)

The ProcessStatus is basically a dataclass that contains the status information of a running process.

The available information is the following:

pid: the process identifier
uptime: the process up-time as a floating point number expressed in seconds
uuid: the UUID1 for this process
memory info: memory information on the process
cpu usage, percentage and count (number of physical cores)

The status will always be updated before returning or printing.

Parameters:

Name	Type	Description	Default
`metrics_prefix`	`Optional[str]`	the prefix that identifies the process for which these metrics are gathered.	`None`

Methods:

Name	Description
`as_dict`	Returns all process information as a dictionary.
`update`	Updates those values that change during execution, like memory usage, number of
`update_metrics`	Updates the metrics that are taken from the psutils module.

as_dict ¶

as_dict()

Returns all process information as a dictionary.

This runs the update() method first to bring the numbers up-to-date.

update ¶

update()

Updates those values that change during execution, like memory usage, number of connections, ...

This call will also update the metrics!

Returns:

Type	Description
`ProcessStatus`	the ProcessStatus object, self.

update_metrics ¶

update_metrics()

Updates the metrics that are taken from the psutils module.

The following metrics are never updated since they are not changed during a process execution:

PSUTIL_NUMBER_OF_CPU
PSUTIL_PID

SubProcess ¶

SubProcess(
    name,
    cmd,
    args=None,
    shell=False,
    stdout=DEVNULL,
    stderr=DEVNULL,
)

A SubProcess that is usually started by the ProcessManager.

Example:

proc = SubProcess("MyApp", [sys.executable, "-m", "egse.<module>"])
proc.execute()

Methods:

Name	Description
`execute`	Execute the sub-process.
`exists`	Checks if the sub-process exists by checking if its process ID exists.
`is_running`	Check if this process is still running.
`quit`	Send a request to quit to the process.
`reap_children`	Tries hard to terminate and ultimately kill all the children of this process.
`returncode`	Check if the sub-process is terminated and return its return code or None when the process

execute ¶

execute()

Execute the sub-process.

Returns:

Type	Description
`bool`	True if the process could be started, False on error.

exists ¶

exists()

Checks if the sub-process exists by checking if its process ID exists.

Returns:

Type	Description
`bool`	True if the sub-process exists.

is_running ¶

is_running()

Check if this process is still running.

checks if process exists
checks if process is not a zombie and is not dead

Returns:

Type	Description
`bool`	True if the process is running.

quit ¶

quit()

Send a request to quit to the process.

This will first send a SIGTERM signal to the process, if that fails, a SIGKILL will be sent.

Returns:

Type	Description
`int`	0 if the process and its sub-processes are all terminated. Will return > 0 to indicate the number of processes that survived the SIGKILL.

reap_children ¶

reap_children(timeout=3)

Tries hard to terminate and ultimately kill all the children of this process.

This will first send a SIGTERM signal to the process, if that fails, a SIGKILL will be sent.

Returns:

Type	Description
`int`	0 if the process and its sub-processes are all terminated. Will return > 0 to indicate the number of processes that survived the SIGKILL.

returncode ¶

returncode()

Check if the sub-process is terminated and return its return code or None when the process is still running.

get_process_info ¶

get_process_info(
    items, contains=True, case_sensitive=False
)

Loops over all running processes and tries to match each item in 'items' to the command line of the process. Any process where all 'items' can be matched will end up in the response.

Returns a list with the process info (PID, cmdline, create_time) for any processes where all 'items' match the process command line. An empty list is returned when not 'all the items' match for any of the processes.

Examples:

>>> get_process_info(items=["feesim"])
[
    {
        'pid': 10166,
        'cmdline': [
            '/Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python',
            '/Users/rik/git/plato-common-egse/venv38/bin/feesim',
            'start',
            '--zeromq'
        ],
        'create_time': 1664898231.915995
    }
]

>>> get_process_info(items=["dpu_cs", "--zeromq"])
[
    {
        'pid': 11595,
        'cmdline': [
            '/Library/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python',
            '/Users/rik/git/plato-common-egse/venv38/bin/dpu_cs',
            'start',
            '--zeromq'
        ],
        'create_time': 1664898973.542281
    }
]

Parameters:

Name	Type	Description	Default
`items`	`List[str] \| str`	a string or a list of strings that should match command line items	required
`contains`	`bool`	if True, the match is done with 'in' otherwise '=='	`True`
`case_sensitive`	`bool`	if True, the match shall be case-sensitive	`False`

Returns:

Type	Description
`List`	A list of process info entries.

is_process_running ¶

is_process_running(
    items,
    contains=True,
    case_sensitive=False,
    as_list=False,
)

Check if there is any running process that contains the given items in its commandline.

Loops over all running processes and tries to match all items in the 'items' argument to the command line of the process. If all 'items' can be matched to a process, the function returns the PID of that process.

By default, only the first matching process PID is returned. If as_list=True then all mathing process PIDs are returned as a list.

Parameters:

Name	Type	Description	Default
`items`	`List[str] \| str`	a string or a list of strings that should match command line parts	required
`contains`	`bool`	if True, the match is done with 'in' otherwise '==' [default: True]	`True`
`case_sensitive`	`bool`	if True, the match shall be case-sensitive [default: False]	`False`
`as_list`	`bool`	return the PID of all matching processes as a list [default: False]	`False`

Returns:

Type	Description
`int \| List[int]`	The PID(s) if there exists a running process with the given items, 0 or [] otherwise.

list_processes ¶

list_processes(
    items,
    contains=True,
    case_sensitive=False,
    verbose=False,
)

Returns and optionally prints the processes that match the given criteria in items.

Parameters:

Name	Type	Description	Default
`items`	`List[str] \| str`	a string or a list of strings that should match command line parts	required
`contains`	`bool`	if True, the match is done with 'in' otherwise '==' [default: True]	`True`
`case_sensitive`	`bool`	if True, the match shall be case-sensitive [default: False]	`False`
`verbose`	`bool`	if True, the processes will be printed to the console	`False`

Returns:

Type	Description
`list[dict]`	A list of dictionaries for the matching processes. The dict contains the 'pid', 'status' and 'cmdline' of a process.

list_zombies ¶

list_zombies()

Returns a list of zombie processes.

A zombie process, also known as a defunct process, is a process that has completed its execution but still has an entry in the process table. This happens when a child process terminates, but the parent process hasn't yet read its exit status by using a system call like wait(). As a result, the process is "dead" (it has completed execution) but hasn't been "reaped" or removed from the system's process table.

A zombie process can not be killed with SIGKILL because it's already dead, and it's only removed when the parent process reads their exit status or when the parent process itself terminates.

A zombie process does not block ports, so it's mostly harmless and will disappear when the parent process terminates.

Returns:

Type	Description
`list[dict]`	A list of dictionaries with information on the zombie processes. The dict contains the 'pid', 'name', and 'cmdline' of the zombie process.