architecture · Jun 2025 – Present

Event-Driven Services for High-Volume Energy & EV Data

Designed and scaled event-driven services on AWS EventBridge and SQS with idempotent consumers and retries, and led the Cassandra migration from Astyanax to the DataStax driver.

Role: Event-driven service design, idempotent consumers, and lead on the Cassandra driver migration
Team: Backend engineering
Scope: High-volume concurrent event streams — EV telematics, managed charging, and core energy data — on Bidgely's B2B energy SaaS platform
Level: Software Engineer

Cassandra client library

AstyanaxDataStax

Method: Led the migration of the Cassandra data-access layer across backend services off the legacy Astyanax client to the supported DataStax driver.
Window: during tenure

Event delivery handling

duplicates unsafeidempotent

Method: Design property of the consumers: at-least-once delivery from SQS/EventBridge is handled with idempotent processing and retry on transient failure.
Window: by design

Core services

legacy clientstreamlined

Method: Qualitative outcome of the driver migration and the streamlining of the core microservices and APIs built on top of it.
Window: post-migration

Context

Bidgely is a B2B energy SaaS platform used by utilities. Behind it sit EV telematics, managed charging, and core energy workflows that produce large, bursty, concurrent streams of device data — exactly the kind of workload that punishes tightly-coupled synchronous services.

My part: designing and scaling the event-driven services that process those streams, and leading the migration of our Cassandra data-access layer onto a supported driver.

Why event-driven

If services call each other synchronously, a slow or failing downstream pushes back-pressure all the way to ingestion, and a traffic burst has nowhere to go but the floor. Modeling the data as events on AWS EventBridge and SQS decouples producers from consumers: bursts buffer in durable queues, consumers scale on their own, and a transient downstream failure becomes a retry instead of a cascade.

The cost is eventual consistency and harder end-to-end tracing — a trade worth making for a high-volume, bursty workload where reliability is the point.

Idempotency is not optional

Queue delivery is at-least-once. Retries, visibility-timeout redeliveries, and replays all mean the same event can show up twice. So every consumer is built to be idempotent — keyed on a stable event identity, so processing the same event twice has the same effect as once. That's what makes retries safe, and it's non-negotiable for anything that feeds energy or charging data downstream.

The Cassandra driver migration

The services talked to Cassandra through Astyanax — a client that's deprecated and no longer maintained. Every month on it accrued risk: no upstream fixes, harder onboarding, growing distance from current Cassandra. I led the migration to the supported DataStax driver, and used it as the moment to streamline the core microservices and APIs that sat on top.

The subtle part of a client swap isn't the API surface; it's the defaults. Consistency levels, retry policies, and timeouts differ between clients, and a silent change in any of them changes how your queries behave under failure. I verified read/write behavior matched the old client before cutting each service over.

Outcomes

High-volume concurrent event streams flow through resilient, idempotent event-driven services, and the data-access layer runs on the supported DataStax driver — better performance, better stability, and less standing platform risk.

Decision Records

DR-001 · architecture

Modeling high-volume device and energy data as events on EventBridge and SQS — rather than synchronous service calls — is what lets the system absorb spikes and recover from partial failures on its own

Context

EV telematics, managed charging, and core energy workflows generate large, bursty, concurrent streams of device data. Synchronous request/response coupling means any slow or failing downstream backs pressure all the way up to ingestion.

Options considered

✗
Synchronous service-to-service calls
Tight coupling. A slow or failing downstream propagates failure upstream, and bursts have nowhere to buffer — you drop data or fall over under load.
✓
Event-driven services with durable queues and an event bus
EventBridge routes events, SQS buffers them durably. Producers and consumers are decoupled, consumers scale independently, and transient downstream failures retry instead of cascading.

Tradeoffs

Eventual consistency and harder end-to-end tracing — you trade the simplicity of a synchronous call for resilience and scale. Worth it for a high-volume, bursty workload where reliability is the priority.

Outcome

High-volume concurrent event streams are processed with resilient failure handling, and bursts are absorbed by the queues instead of the services.

DR-002 · architecture

Because queue delivery is at-least-once, idempotent consumers plus retries are the only way to get correct results — not an optimization

Context

SQS and EventBridge deliver at least once. Retries, redeliveries after a visibility timeout, and replays all mean the same event can arrive more than once. For energy and charging data, processing an event twice can't be allowed to change the result.

Options considered

✗
Assume exactly-once delivery and process events as they arrive
The queues don't guarantee it. Duplicates are a matter of when, not if — and silent double-processing corrupts downstream data.
✓
Idempotent consumers keyed on a stable event identity, with retries for transient failures
Processing the same event twice has the same effect as once, so retries become safe. Transient failures get retried instead of dropped.

Tradeoffs

Every consumer needs a stable idempotency key and a way to detect already-applied events, which adds design work to each new service. Cheap compared with chasing duplicate-processing bugs in production.

Outcome

Consumers handle redeliveries and retries safely, so failure recovery is automatic rather than a manual clean-up job.

DR-003 · architecture

Moving the data-access layer to the supported DataStax driver — and streamlining the core microservices and APIs on top of it — improved performance and long-term stability

Context

Backend services accessed Cassandra through the legacy Astyanax client, which is deprecated and unmaintained. Staying on it meant accumulating risk: no upstream fixes, harder onboarding, and growing distance from current Cassandra features.

Options considered

✗
Stay on the Astyanax client
Deprecated and unmaintained — no upstream fixes, and the gap to modern Cassandra features only widens. Accumulating risk for no benefit.
✓
Migrate to the supported DataStax driver
Actively maintained, better query reliability, and the long-term-supported path. Removes a standing platform risk and gave us the chance to streamline the services that sat on it.

Tradeoffs

The migration touches every service that talks to Cassandra and has to be done carefully to avoid query-behavior regressions. Worth it because the alternative is indefinite tech debt on a dead client.

Outcome

Backend services moved onto the DataStax driver, and the core microservices and APIs were streamlined in the process — improving performance and stability.

Where I Got It Wrong

Failure 1

During the driver migration, default consistency-level and retry-policy differences between Astyanax and the DataStax driver meant query behavior could change subtly under the services if taken at face value

Cost: Extra verification work to confirm read/write behavior matched the old client before cutting services over

Lesson: When swapping a database client, treat the default consistency, retry, and timeout settings as part of the contract — they differ between clients, and a silent change in them is exactly the kind of bug that only shows up under failure

Long-Term Impact

Event-driven services with idempotent consumers and the supported Cassandra driver are the foundations the team builds new energy and telematics workflows on.