events module

class events.EventProcessor(process_cadence=0.1, request_timeout=0.2, num_messages=10, kafka_server=None, input_topic=None, tag='default', send_heartbeats=False, heartbeat_cadence=None, heartbeat_topic=None)

Bases: object

Base class for processing events via Kafka.

Parameters

kafka_serverstr

the host:port combination to connect to the Kafka broker

input_topicstr

the name of the input topic

process_cadencefloat

maximum rate at which data is processed, defaults to 0.1s

request_timeoutfloat

timeout for requesting messages from a topic, defaults to 0.2s

num_messagesint

max number of messages to process per cadence, defaults to 10

tagstr

a nickname for the instance, defaults to ‘default’

send_heartbeatsbool

send periodic heartbeat messages to Kafka for monitoring

heartbeat_cadencefloat

cadence on which to write heartbeat messages to Kafka

heartbeat_topicstr

Kafka topic to send heartbeats to

catch(signum, frame)

Shuts down the event processor gracefully before exiting.

fetch()

Fetch for messages from a topic and processes them.

finish()

Finish remaining events when stopped and/or shutting down.

NOTE: Derived classes may implement this if desired.

handle()

Handles ingested events.

NOTE: Derived classes need to implement this.

heartbeat()

Send heartbeat messages to Kakfa to monitor the health of this process.

ingest(message)

Ingests a single event.

NOTE: Derived classes need to implement this.

process()

Processes events at the specified cadence.

start()

Starts the event loop.

stop()

Stops the event loop.

events.append_args(parser)

Append event processing specific options to an ArgumentParser instance.