Failure</th>	Handled By</th></tr></thead>
DB commit succeeds, broker publish fails</strong></td>	Relay retries on next cycle (automatic recovery)</td></tr>
DB transaction rolls back</strong></td>	Event row never inserted; no phantom message</td></tr>
Relay crashes mid-cycle</strong></td>	Next cycle picks up PENDING events (idempotent)</td></tr>
Broker is down for hours</strong></td>	Events remain PENDING in DB; relay publishes when broker recovers</td></tr>
Event permanently malformed</strong></td>	Marked FAILED after max-retries; operator reviews error_message</td></tr>
Duplicate delivery</strong></td>	Consumer must deduplicate using event id (UUID)</td></tr> </tbody></table> Monitor the outbox table for events stuck in PENDING state (age > 5× relay interval) or accumulating in FAILED state. These are early warning signs of broker issues or payload corruption.</p> #</a> Implementation Guide</h2> Prerequisites:</strong></p> Spring Boot 3.x or 4.x</li> Java 17+</li> Spring Data JPA with PostgreSQL (or compatible database with UUID support)</li> Apache Kafka or RabbitMQ</li> Gradle or Maven</li> </ul> Add the starter to build.gradle.kts:</strong></p> dependencies </span>{</span></span> implementation</span>(</span>"</span>io.github.saumilp.starters:spring-boot-starter-outbox:1.0.0</span>"</span>)</span></span> implementation</span>(</span>"</span>org.springframework.kafka:spring-kafka:3.3.0</span>"</span>)</span></span> //</span> or:</span></span> //</span> implementation("org.springframework.boot:spring-boot-starter-amqp:3.2.0")</span></span> }</span></span></code></pre> Step 1: Create the Outbox Table</strong></p> Run the SQL DDL above against your PostgreSQL database. The table should be created during initial schema setup or via Flyway/Liquibase migrations.</p> Step 2: Configure the Starter</strong></p> In application.yml</code>:</p> s</span>pring</span>:</span></span> o</span>utbox</span>:</span></span> e</span>nabled</span>:</span> true</span></span> r</span>elay-interval-ms</span>:</span> 5000</span></span> m</span>ax-retries</span>:</span> 3</span></span> b</span>ackoff-multiplier</span>:</span> 2.0</span></span> k</span>afka</span>:</span></span> b</span>ootstrap-servers</span>:</span> l</span>ocalhost:9092</span></span> t</span>opic-prefix</span>:</span> o</span>utbox.</span></span></code></pre> Step 3: Implement Your Service</strong></p> @</span>Service</span></span> public</span> class</span> OrderService</span> {</span></span> private</span> final</span> OrderRepository</span> orderRepository</span>;</span></span> private</span> final</span> OutboxPublisher</span> outboxPublisher</span>;</span></span> </span> @</span>Transactional</span></span> public</span> Order</span> createOrder</span>(</span>CreateOrderRequest</span> req</span>)</span> {</span></span> Order</span> order</span> =</span> orderRepository</span>.</span>save</span>(</span>new</span> Order</span>(</span>req</span>)</span>)</span>;</span></span> </span></span> outboxPublisher</span>.</span>publish</span>(</span></span> new</span> OrderCreatedEvent</span>(</span></span> order</span>.</span>getId</span>(</span>)</span>,</span></span> order</span>.</span>getCustomerId</span>(</span>)</span>,</span></span> order</span>.</span>getTotal</span>(</span>)</span></span> )</span></span> )</span>;</span></span> </span></span> return</span> order</span>;</span></span> }</span></span> }</span></span></code></pre> The @Transactional</code> annotation ensures both the order AND the outbox event are persisted atomically.</p> Step 4: Consuming Events</strong></p> Consumers must deduplicate by event ID to handle at-least-once delivery:</p> @</span>Service</span></span> public</span> class</span> OrderEventConsumer</span> {</span></span> private</span> final</span> OrderEventRepository</span> eventRepository</span>;</span></span> </span> @</span>KafkaListener</span>(</span>topics</span> =</span> "</span>outbox.ORDER_CREATED</span>"</span>)</span></span> public</span> void</span> handleOrderCreated</span>(</span>OrderCreatedEvent</span> event</span>)</span> {</span></span> //</span> Check if already processed</span></span> if</span> (</span>eventRepository</span>.</span>existsByEventId</span>(</span>event</span>.</span>getId</span>(</span>)</span>)</span>)</span> {</span></span> return</span>;</span></span> }</span></span> </span></span> //</span> Process the event</span></span> processOrder</span>(</span>event</span>)</span>;</span></span> </span></span> //</span> Record that we processed it</span></span> eventRepository</span>.</span>save</span>(</span>new</span> ProcessedEvent</span>(</span>event</span>.</span>getId</span>(</span>)</span>)</span>)</span>;</span></span> }</span></span> }</span></span></code></pre> Consumer Idempotency Patterns:</strong></p> In-memory deduplication (low-traffic):</strong> Store event IDs in a HashSet. Simple but lost on restart.</p> </li> Database deduplication (recommended):</strong> Store processed event IDs in a database table with unique constraint. Survives restarts.</p> </li> Idempotent business operations (best):</strong> Make the operation itself idempotent. Use event ID as a key. If the operation runs twice, it’s safe.</p> </li> Database constraints (clever):</strong> Use UNIQUE constraint on (event_id, business_key). The second insert fails safely.</p> </li> </ol> #</a> Infrastructure and Deployment</h2> Docker Compose for local development:</strong></p> v</span>ersion</span>:</span> '</span>3.8</span>'</span></span> s</span>ervices</span>:</span></span> p</span>ostgres</span>:</span></span> i</span>mage</span>:</span> p</span>ostgres:16</span></span> e</span>nvironment</span>:</span></span> P</span>OSTGRES_DB</span>:</span> o</span>utbox_demo</span></span> P</span>OSTGRES_PASSWORD</span>:</span> p</span>ostgres</span></span> p</span>orts</span>:</span></span> -</span> "</span>5432:5432</span>"</span></span> v</span>olumes</span>:</span></span> -</span> p</span>ostgres_data:/var/lib/postgresql/data</span></span> </span> k</span>afka</span>:</span></span> i</span>mage</span>:</span> c</span>onfluentinc/cp-kafka:7.6.0</span></span> d</span>epends_on</span>:</span></span> -</span> z</span>ookeeper</span></span> p</span>orts</span>:</span></span> -</span> "</span>9092:9092</span>"</span></span> e</span>nvironment</span>:</span></span> K</span>AFKA_BROKER_ID</span>:</span> 1</span></span> K</span>AFKA_ZOOKEEPER_CONNECT</span>:</span> z</span>ookeeper:2181</span></span> K</span>AFKA_ADVERTISED_LISTENERS</span>:</span> P</span>LAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092</span></span> K</span>AFKA_AUTO_CREATE_TOPICS_ENABLE</span>:</span> "</span>true</span>"</span></span> </span> z</span>ookeeper</span>:</span></span> i</span>mage</span>:</span> c</span>onfluentinc/cp-zookeeper:7.6.0</span></span> p</span>orts</span>:</span></span> -</span> "</span>2181:2181</span>"</span></span> e</span>nvironment</span>:</span></span> Z</span>OOKEEPER_CLIENT_PORT</span>:</span> 2181</span></span> </span> v</span>olumes</span>:</span></span> p</span>ostgres_data</span>:</span></span></code></pre> Testing the flow:</strong></p> Start services: `docker-compose up</code></li>` `Create an order: curl -X POST http://localhost:8080/orders -H "Content-Type: application/json" -d '{"customerId":"cust-123", "amount":99.99}'</code></li>` `Check outbox table: SELECT * FROM outbox_events;</code></li>` `Verify Kafka topic has the event: kafka-console-consumer.sh --topic outbox.ORDER_CREATED --bootstrap-servers localhost:9092</code></li>` `Verify consumer processed it: SELECT * FROM processed_events;</code></li> </ol> CI/CD Pipeline (GitHub Actions):</strong></p>`n</span>ame</span>:</span> T</span>est Outbox Starter</span></span> on</span>:</span> [</span>p</span>ush</span>,</span> p</span>ull_request</span>]</span></span> </span> j</span>obs</span>:</span></span> t</span>est</span>:</span></span> r</span>uns-on</span>:</span> u</span>buntu-latest</span></span> s</span>ervices</span>:</span></span> p</span>ostgres</span>:</span></span> i</span>mage</span>:</span> p</span>ostgres:16</span></span> e</span>nv</span>:</span></span> P</span>OSTGRES_PASSWORD</span>:</span> p</span>ostgres</span></span> o</span>ptions</span>:</span> ></span>-</span></span> --health-cmd pg_isready</span></span> --health-interval 10s</span></span> --health-timeout 5s</span></span> --health-retries 5</span></span> </span> s</span>teps</span>:</span></span> -</span> u</span>ses</span>:</span> a</span>ctions/checkout@v3</span></span> -</span> u</span>ses</span>:</span> a</span>ctions/setup-java@v3</span></span> w</span>ith</span>:</span></span> j</span>ava-version</span>:</span> 21</span></span> d</span>istribution</span>:</span> t</span>emurin</span></span> -</span> r</span>un</span>:</span> .</span>/gradlew test --info</span></span></code></pre> #</a> Key Takeaways</h2> For Architects:</strong> The Outbox pattern is NOT a substitute for event sourcing. Use it for reliable event publishing in transaction-based services. For temporal queries, event replay, or audit trails, event sourcing is the better choice.</p> For Developers:</strong> Remember: the pattern provides at-least-once delivery, so consumers MUST be idempotent. Use the event ID as your deduplication key. Test both success and failure paths (relay down, broker down, transaction rollback).</p> For DevOps:</strong> Monitor the outbox table cardinality and relay latency. Set up alerts for FAILED events accumulating. The relay is stateless, so you can run multiple instances for redundancy.</p> Best Practices:</strong></p> Always use event IDs for consumer deduplication</li> Make business operations idempotent, not just event deduplication</li> Set reasonable max-retries (usually 3-5)</li> Use exponential backoff for retries</li> Log failed events with full error messages</li> Partition by aggregateId for ordering guarantees</li> Never delete from outbox table; archive to cold storage after 30 days</li> </ul> Kafka vs RabbitMQ Trade-offs:</strong></p> Aspect</th> Kafka</th> RabbitMQ</th></tr></thead> Ordering</strong></td> Per-partition (use aggregateId)</td> Per-queue (declare exclusive queues)</td></tr> At-least-once</strong></td> Built-in with consumer groups</td> Requires manual ACKs</td></tr> Retention</strong></td> Days/weeks (configurable)</td> Until consumed</td></tr> Scaling</strong></td> Horizontal (partitions)</td> Vertical plus clustering</td></tr> Complexity</strong></td> Moderate (Zookeeper/KRaft)</td> Simple setup</td></tr> </tbody></table> For the Outbox pattern, Kafka’s partition-based ordering is ideal. For RabbitMQ, use topic exchanges with routing keys bound to dedicated queues.</p> #</a> References and Further Reading</h2> Transactional Outbox Pattern:</strong> https://microservices.io/patterns/data/transactional-outbox.html</li> Spring Boot Starter Outbox:</strong> https://github.com/saumilpatel/spring-boot-starter-outbox</li> Event-Driven Architecture:</strong> https://www.oreilly.com/library/view/building-event-driven-microservices/</li> Kafka at-least-once semantics:</strong> https://kafka.apache.org/documentation/#consumerconfigs_enable.auto.commit</li> Spring Data JPA:</strong> https://spring.io/projects/spring-data-jpa</li> PostgreSQL UUID type:</strong> https://www.postgresql.org/docs/current/datatype-uuid.html</li> </ul> #</a> Troubleshooting</h2> Q: Events are stuck in PENDING status</strong> A: Check if the relay is running (logs should show “Polling outbox…”). Verify broker connectivity. Check error_message column for details.</p> Q: “Transaction already rolled back” error when calling publish()</strong> A: OutboxPublisher must be called within a @Transactional method. The transaction must be active and not already marked for rollback.</p> Q: Duplicate messages in the broker</strong> A: The pattern guarantees at-least-once. Consumers MUST deduplicate by event ID. This is not a bug; it’s the contract.</p> Q: Relay is taking too long (> relay-interval-ms)</strong> A: Increase relay-interval-ms or add database indexes. If outbox_events table is huge, partition it by created_at or status.</p> Q: How do I handle event schema changes?</strong> A: Version your event payloads (add a “version” field). Consumers should support multiple versions. Use JSON Schema or Protocol Buffers for validation.</p> Q: Can I use this with RabbitMQ?</strong> A: Yes. Configure rabbitmq section instead of kafka in application.yml. The pattern is broker-agnostic.</p> Colophon: how this site is built 2026-06-28T00:00:00+00:00 This site is a static, dependency-light technical publication.</p> Generator:</strong> Zola</a> — a single fast Rust binary, no Node build pipeline.</li> Templates:</strong> custom Tera</a> partials and macros.</li> Styles:</strong> a hand-written SCSS design system compiled by Zola, driven by CSS custom properties (light + dark).</li> Search:</strong> Pagefind</a> — a fully static search index generated after the build.</li> Interactions:</strong> a few tiny vanilla-JS modules (theme toggle, copy-code, mobile nav, search, TOC) — no framework.</li> Hosting:</strong> GitHub Pages, built and deployed by GitHub Actions.</li> </ul> No client-side framework, no CDN dependency, no external analytics.</p> Java Review MCP Server 2025-02-01T00:00:00+00:00 The problem</h2> AI-assisted code review requires understanding both the syntax and the broader system design context. Generic code review tools miss enterprise-specific concerns: transaction boundaries, observability instrumentation, context propagation patterns, and domain-driven design principles. Integrating code review into a developer’s workflow without context switching is difficult.</p> Approach</h2> Built an MCP (Model Context Protocol) server that exposes a Java project’s structure and analysis capabilities to Claude, enabling context-aware code reviews. The server parses the codebase, builds an AST-based understanding of packages, classes, and dependencies, then provides Claude with structured access to perform deep reviews.</p> Key capabilities:</strong></p> Project structure analysis and dependency graph visualization</li> Class and method-level code analysis with architectural context</li> Transaction boundary detection and verification</li> Observability instrumentation pattern recognition</li> Security vulnerability pattern matching</li> Architecture constraint validation</li> </ul> Trade-offs</h2> Full project analysis over isolated review:</strong> Slower startup but enables holistic review feedback. Caches analysis to reduce latency on repeated reviews.</li> Claude as reviewer over specialized linters:</strong> More context-aware and human-friendly feedback, but requires language model access and costs per review.</li> MCP protocol over custom API:</strong> Ensures interoperability with Claude and other tools, but adds protocol overhead.</li> </ul> Outcome</h2> Seamless integration of AI-assisted code review into Claude workflows. Teams get sophisticated architectural review feedback without leaving their editor or IDE. Particularly valuable for enforcing enterprise patterns across distributed teams and mentoring junior engineers on system design principles.</p> Dev machine setup for Flutter App development 2024-11-16T00:00:00+00:00 </p> Setting up Flutter</a> on macOS can be staightforward process when using Homebrew</a>, a popular package manager for macOS. Flutter, a powerful open-source UI software development toolkit by Google, enables developers to build natively compiled mobile, web, and desktop applications from a single codebase. In this guide, we’ll walk you through the necessary steps to install and configure Flutter on macOS system using Homebrew. Additionally, we will cover the installation of essential tools like Android Studio and Xcode, update Ruby, and set up CocoaPods to ensure a smooth development experience. Rust Learning Lab 2024-06-15T00:00:00+00:00 The problem</h2> Learning Rust requires understanding both the language syntax and the systems programming concepts it enables. Most tutorials focus on syntax, while real-world systems programming involves memory safety, concurrency patterns, and performance optimization. There’s a gap between “Hello, World!” and building production-grade systems in Rust.</p> Approach</h2> Created a comprehensive learning project that progresses from fundamentals to advanced systems programming. The lab covers ownership and borrowing through practical examples, explores concurrency patterns with real-world scenarios, and includes deep dives into performance optimization.</p> Key areas:</strong></p> Core concepts:</strong> Ownership, borrowing, lifetimes, and the type system with runnable examples</li> Systems programming:</strong> Memory management, unsafe code patterns, and performance profiling</li> Concurrency:</strong> Threading, async/await, channels, and lock-free patterns</li> Algorithms & data structures:</strong> Implementations with trade-off analysis</li> Performance:</strong> Benchmarking, profiling, and optimization techniques</li> </ul> Trade-offs</h2> Breadth over depth:</strong> Covers many topics at intermediate level rather than mastering one deeply. Designed as a learning progression, not a reference.</li> Examples over libraries:</strong> Most implementations are from scratch to teach concepts, not production-ready code.</li> Educational comments over minimal code:</strong> Heavy documentation and explanations to aid learning, which would be excessive in production code.</li> </ul> Outcome</h2> A self-directed learning resource for engineers transitioning to Rust or deepening their systems programming knowledge. Particularly valuable for those coming from higher-level languages who need to understand memory safety and performance characteristics. Serves as both a learning guide and a reference for Rust patterns in real systems.</p> Learning to fly with Rust 2024-04-01T00:00:00+00:00 </p> Recently I updated my blog-site with Zola</strong></a>, and then I started thinking to host Rust application on cloud providers. There are multiple CSPs ( Cloud Service Providers</em> ) enabling Rust enthusiasts Developers (also known as “Rustaceans”) to host web-apps on cloud. Zola quickstart 2024-03-19T00:00:00+00:00 This post is one of the first one after a long time, I am evaluating Zola</code></a></strong> as replacement of Jekyll, and thought of creating a blog for step-by-step site building.</p> Whether you like Rust and SSG or you do not know any of these, if you wish to create a website or a blog, Zola</code> is made for you! We are going to discover here the basic concepts.… </p> Mastodon Toot Client 2024-02-01T00:00:00+00:00 The problem</h2> Mastodon’s official apps don’t support scheduling or file-based content. Automating posts for a personal account — say, a weekly architecture tip from a text file — required either a Python script with fragile state management or a paid third-party service.</p> Approach</h2> Built a stateless Rust CLI: reads a text file of toots (one per line), posts the next unposted one to the Mastodon API, and marks it done by moving the line to a done</code> file. Runs as a GitHub Actions cron job — no server required, credentials stored as Actions secrets.</p> Trade-offs</h2> Stateless file-append over a database:</strong> Simpler deployment and easier to audit. The “done” file doubles as a history log.</li> Rust over Python:</strong> Compile-time guarantees matter when the binary runs unattended. The async Tokio runtime handles the HTTP call without blocking.</li> </ul> Outcome</h2> A zero-dependency automation that posts on a schedule without any hosted infrastructure. The approach generalises to any API-driven content automation with similar stateless patterns.</p> Spring Boot Starters 2024-01-15T00:00:00+00:00 The problem</h2> Building microservices requires solving the same problems repeatedly: atomic event publishing, context propagation across async boundaries, cardinality control in observability pipelines, and idempotent message consumption. Each team implements these differently, leading to fragmented traces, inconsistent retry logic, and high-cardinality metrics explosions.</p> Approach</h2> Created a suite of focused Spring Boot starters that encode opinions about how to solve these problems consistently across a microservice mesh. Each starter is independently deployable and composable, allowing teams to adopt one, several, or all without vendor lock-in.</p> Key starters:</strong></p> spring-boot-starter-outbox:</strong> Transactional outbox pattern for reliable event publishing with automatic relay and retry logic</li> spring-boot-starter-observability:</strong> Enforced context propagation, cardinality controls, and extension points for custom instrumentation</li> spring-boot-starter-idempotency:</strong> Consumer-side deduplication with pluggable backends (in-memory, Redis, database)</li> </ul> Trade-offs</h2> Auto-configuration over manual setup:</strong> Faster onboarding but requires trust in the starter’s opinions. Overrideable via @ConditionalOnMissingBean</code>.</li> Outbox relay over broker transactions:</strong> Avoids distributed transaction complexity but introduces a scheduled component that must be monitored.</li> Platform-level observability over per-team instrumentation:</strong> Ensures consistency but loses flexibility for teams with unique requirements.</li> </ul> Outcome</h2> A production-proven library reducing time-to-reliability for event-driven systems. Eliminates boilerplate around event publishing, context propagation, and idempotent processing. Used across fintech and telecom microservice architectures handling millions of events per day.</p> draw.io Architecture Libraries 2024-01-01T00:00:00+00:00 The problem</h2> Architecture diagrams in most teams suffer from inconsistency — each engineer invents their own notation, colour scheme, and level of abstraction. Reviews spend more time decoding the diagram than discussing the decision.</p> Approach</h2> Built a curated set of draw.io shape libraries covering C4 model levels (Context, Container, Component), AWS service icons, and a custom notation layer for domain events and async flows. Libraries are XML files that can be imported directly into draw.io desktop or app.diagrams.net with one click.</p> Trade-offs</h2> Chose draw.io over Mermaid/PlantUML:</strong> Draw.io diagrams are visual-first and accessible to non-engineers. Mermaid is better for version control but harder to layout precisely.</li> Static XML over dynamic generation:</strong> Keeps the library dependency-free and usable offline. The trade-off is that updating icons requires editing XML by hand.</li> </ul> Outcome</h2> The libraries are used in personal architecture documentation and have been shared with colleagues for review. The C4 layer in particular cuts diagram preparation time significantly by providing pre-built component shapes with consistent colour coding.</p> Enterprise Spring Patterns 2023-09-01T00:00:00+00:00 The problem</h2> Spring Boot makes the happy path easy, but enterprise systems have edge cases: transactional outbox patterns, idempotent consumers, domain event publishing, and layered service boundaries. These topics are scattered across blog posts with inconsistent advice.</p> Approach</h2> Built a reference application demonstrating enterprise patterns with real infrastructure: PostgreSQL for persistence, Kafka for event streaming, and Redis for idempotency keys. Each pattern is isolated in its own package with an integration test that uses Testcontainers.</p> Trade-offs</h2> Testcontainers over H2:</strong> H2 in-memory DB hides PostgreSQL-specific behaviour. Testcontainers is slower to start but catches real issues.</li> Outbox pattern over direct Kafka publish:</strong> Ensures at-least-once delivery without distributed transactions.</li> </ul> Outcome</h2> A self-contained reference for backend engineers migrating from monolith Spring MVC to event-driven architectures. Covers the gap between “Hello World” Spring Boot tutorials and production-grade system design.</p> Java Design Patterns 2023-06-01T00:00:00+00:00 The problem</h2> Design patterns are often taught with UML diagrams and abstract descriptions. Real engineers need runnable code they can step through, modify, and connect to frameworks they already use (Spring, CDI, etc.).</p> Approach</h2> Each pattern gets its own Maven module: a minimal implementation, a Spring Boot integration where it makes sense, and JUnit 5 tests that assert the expected behaviour. Patterns are documented with a short “when to use / when not to” note rather than a textbook definition.</p> Trade-offs</h2> Maven multi-module over Gradle:</strong> Heavier to set up but more familiar to enterprise Java developers who are the primary audience.</li> Spring integration optional:</strong> Not every pattern maps naturally to Spring — forced integration would produce misleading examples.</li> </ul> Outcome</h2> A living reference that maps GoF patterns to real Java 21 idioms (records, sealed classes, switch expressions) and framework-level equivalents in Spring Boot 3.</p> gh-stats 2023-03-01T00:00:00+00:00 The problem</h2> GitHub contribution graphs and repository statistics are useful for portfolios and dashboards, but embedding them requires either heavy JavaScript libraries or manual updates. Serverless functions are perfect for this — lightweight, scalable, and stateless — but most implementations over-engineer the solution with unnecessary dependencies.</p> Approach</h2> Built a minimal serverless function that queries the GitHub API and generates clean SVG visualizations. The service is stateless, dependency-light, and optimized for fast response times. Each visualization is generated on-demand, eliminating cache invalidation headaches.</p> Key capabilities:</strong></p> Contribution graph:</strong> Visual timeline of GitHub contributions</li> Repository stats:</strong> Stars, forks, language distribution</li> User stats:</strong> Public profile metrics and activity summaries</li> SVG generation:</strong> No external rendering services — pure SVG generation</li> Caching headers:</strong> Appropriate cache directives for dashboard embedding</li> </ul> Trade-offs</h2> SVG over canvas/images:</strong> SVG is lightweight and infinitely scalable, but rendering is limited to basic shapes and text. No photorealistic charts.</li> On-demand over pre-computed:</strong> Function calls add latency, but eliminate stale data and infrastructure complexity.</li> Minimal dependencies:</strong> Reduces bundle size and attack surface, but requires more custom logic than using charting libraries.</li> </ul> Outcome</h2> A lightweight tool for developers who want GitHub stats on their websites, portfolios, or dashboards without heavy client-side libraries. Particularly useful for status pages, portfolio sites, and internal dashboards where fresh data matters but heavy infrastructure doesn’t.</p> Automatic ID generation in Apache Solr 2015-02-16T00:00:00+00:00 I have been working on Apache Solr for last few months, and have been recieving requirements to speed up query process. As part of the investigation, i found out as retrieved documents’ unique id generation contributes query processing.And hence i have decided to add this post.… </p> XML Schema Design Patterns 2015-02-09T00:00:00+00:00 This might be one of the old topic - specially when lot of developers are working with RESTful Services, but i would like to point out couple of pros and cons.</p> As most of the developers know, XML schema is very powerful, but they lack object-orientation, and are intended to capture a data model rather than an object model. In last one and half decades, there has been few tools available to architects and developers to be able to make use off some of the object orientation principles - like polymorphism for flexibility. But ultimately the whole point of the services must be general enough that it can communicate across languages. Hence i have attempted to put together some information around some of the very well known Design patterns.… </p> Curried Function in Java 2015-01-31T00:00:00+00:00 Currying</strong> is a technique of transforming a function with multiple arguments into a function with just single argument. The single argument is the value of the first argument from the original function and it returns another single argument function. This in turn would take the second original argument and itself return another single argument function. This kind of chaining continues over the arguments of the original. The last in the chain will have access to all the arguments and so can do whatever it needs to do.… </p> Quick Apache Solr Setup 2015-01-31T00:00:00+00:00 </p> It has been long time since I started with setup/configuration related blog post. This post provides steps for setting up developer machine for Apache Solr</a></strong>. Apache Solr is an opensource search engine written in Java, from the Apache Lucene project - with many powerful features</a>. Closures in Java 2015-01-23T00:00:00+00:00 Working on last few projects, i’ve realized that use of closures can come really handy in day-to-day work. Can you imagine writing loops after loops to extract or collect properties from list of custom objects? It can be tiring and painful. And hence, I have decided to add this post - mainly to point out couple of situations where introduction to closures can really ease this pain.… </p>

Machine Mind

Custom Spring Boot Observability Starter: When Every Team's Instrumentation Becomes Your Bottleneck

2026-07-06T00:00:00+00:00

Your telecom carrier processes 2 million call detail records (CDRs) per hour across six microservices: ingestion, validation, enrichment, rating, ledger, and billing. A customer calls complaining about an erroneous charge. You trace their call: it entered the pipeline at 14:32:17 UTC. By 14:32:22, it was marked BILLED. But somewhere between enrichment and rating—a 3-second window—the trace goes dark. Two disjoint traces where there should be one continuous flow.

You open Jaeger. The enrichment service shows the call’s trace ID. The rating service shows a different trace ID for the same call. Context was lost in an async enrichment callback that queued to Kafka. The enrichment team instrumented their callback with Micrometer @Observed</code> but didn’t use ContextPropagatingTaskDecorator</code>. The rating team did. Different approaches, both reasonable-looking, but they broke distributed tracing across team boundaries. Now you’ve spent 90 minutes debugging what should have been a 10-minute investigation.

This is what happens when you leave observability as a per-team concern. Each service adds its own @Observed</code> annotations, uses slightly different context propagation logic, and tunes cardinality differently. The mesh is theoretically observable, but the traces are fragmented, context leaks at async boundaries, and your SLO dashboards show double-counted spans from overlapping instrumentation. When you’re processing millions of CDRs per day, every second of debug delay multiplies cost.

A custom observability starter solves this by making tracing and metrics a platform-level concern. Every service in your mesh gets the same opinion about what to instrument, how to propagate context, how to avoid cardinality explosions, and how to extend it without forking.

#</a>
Why Build vs. Buy</h2>
Vendor-provided auto-instrumentation</strong> (Datadog, New Relic, Splunk) solves observability. Their agents inject bytecode, intercept calls, and emit spans—no code changes needed. You pay per GB and get their UI bundled.</p>
Why we built instead:</strong></p>
The big cloud observability vendors are optimized for black-box applications. They emit a lot</em> of data—every DB query, every HTTP call, every exception. In a telecom context, that cardinality explodes. A CDR (call detail record) processing pipeline might touch a customer ID dimension table on every row. Auto-instrumentation sees that as 2 million independent queries per hour and emits 2 million spans per hour. Your cardinality limits hit in minutes.</p>
You also lose control. Your vendor’s defaults might capture PII or highly sensitive telecom fields (calling line ID, called number, IMSI/IMEI). You can filter in their UI, but that’s reactive. You want to decide at instrumentation time</em> what gets captured. For regulatory compliance (TCPA, GDPR), that matters.</p>
Finally: you own the code</strong>. When a bug in the starter breaks observability for all your services, you’re the on-call engineer. But you’re also the one who can ship a fix in 30 minutes instead of waiting for a vendor release and staging environment validation. For a platform team, that ownership is often the right trade-off.</p>
The downside:</strong> you maintain it. Every minor Spring Boot release, every Micrometer version bump, you validate the starter still works. A critical bug in the starter affects every service in your mesh. The context propagation logic has to cover Reactor, @Async, virtual threads, and ForkJoinPool—that’s a surface area.</p>

#</a>
Architectural Vision</h2>
The starter sits between your application code and the Micrometer Observation + OpenTelemetry layers. It’s not a replacement for either—it’s a wrapper that enforces consistent defaults and provides extension points so teams can override without forking.</p>
graph TD</span></span>
    subgraph App["Application Layer"]</span></span>
        AppCode["Spring Boot App<br/>@Transactional @Observed"]</span></span>
    end</span></span>
    </span></span>
    subgraph Starter["Observability Starter"]</span></span>
        Config["Auto-Configuration<br/>@ConditionalOnMissingBean"]</span></span>
        Context["Context Propagation<br/>Reactor + @Async"]</span></span>
        Extension["Extension Points<br/>Predicate/Filter/Handler"]</span></span>
    end</span></span>
    </span></span>
    subgraph Micrometer["Micrometer & OpenTelemetry"]</span></span>
        MM["Micrometer<br/>Observation API"]</span></span>
        OTel["OpenTelemetry<br/>Exporters"]</span></span>
    end</span></span>
    </span></span>
    subgraph Backends["Observability Backends"]</span></span>
        Jaeger["Jaeger/Tempo"]</span></span>
        Prometheus["Prometheus"]</span></span>
    end</span></span>
    </span></span>
    AppCode --> Config</span></span>
    AppCode --> Extension</span></span>
    Config --> MM</span></span>
    Context --> MM</span></span>
    Extension --> MM</span></span>
    MM --> OTel</span></span>
    OTel --> Jaeger</span></span>
    OTel --> Prometheus</span></span></code></pre>
The starter provides three things:</p>

Sensible defaults</strong> — every service gets the same @Observed</code> setup on HTTP handlers, repository methods, and async tasks.</li>
Context propagation</strong> — trace IDs flow from sync code into Reactor Context</code>, into @Async</code> tasks, even into ForkJoinPool</code> work-stealing threads.</li>
Extension points</strong> — teams can define ObservationPredicates</code> to exclude certain paths, ObservationHandlers</code> to extract custom attributes, and ObservationFilters</code> to rename or suppress spans without editing the starter source.</li>
</ol>

#</a>
Key Design Decisions</h2>
Auto-configuration strategy</strong></p>
Spring Boot’s @ConditionalOnClass</code>, @ConditionalOnMissingBean</code> idiom is the foundation. The starter auto-configures:</p>

ObservationRegistry</code> customization (registers filters, predicates, handlers)</li>
TraceIdGenerator</code> (defaults to OpenTelemetry’s UUID-based generator)</li>
ContextPropagator</code> beans for W3C Trace Context and W3C Baggage</li>
MeterRegistry</code> instrumentation (CPU, memory, GC)</li>
</ul>
Teams can override any bean by defining their own in application.yml</code> or a @Configuration</code> class. No forking required.</p>
The observation lifecycle</strong></p>
Here’s what happens when a span is created:</p>
sequenceDiagram</span></span>
    participant Request as HTTP Request</span></span>
    participant App as App Code</span></span>
    participant Registry as Observation Registry</span></span>
    participant Predicate as Predicate</span></span>
    participant Filter as Filter</span></span>
    participant Handler as Handler</span></span>
    participant Backend as Backend</span></span>
    </span></span>
    Request ->> App: HTTP handler</span></span>
    App ->> Registry: Observation.start()</span></span>
    Registry ->> Predicate: test(name, context)</span></span>
    </span></span>
    alt Predicate rejects</span></span>
        Predicate -->> Registry: false</span></span>
        Registry -->> App: no-op observation</span></span>
    else Predicate accepts</span></span>
        Predicate ->> Registry: true</span></span>
        Registry ->> Filter: process(scope)</span></span>
        Filter ->> Handler: handle(scope)</span></span>
        Handler ->> Backend: emit span</span></span>
        App ->> App: business logic</span></span>
        App ->> Registry: Observation.stop()</span></span>
        Registry ->> Handler: handle(context)</span></span>
        Handler ->> Backend: finalize span</span></span>
        Backend -->> App: done</span></span>
    end</span></span></code></pre>
Context propagation across async boundaries</strong></p>
The hardest part: trace IDs must flow from the original HTTP thread into async tasks. Spring Boot 3.0+ provides TaskExecutor</code> integration via ContextPropagatingTaskDecorator</code>, but you have to opt in per threadpool</em>. The starter makes it automatic.</p>
When you define @Bean public TaskExecutor</code> in your application, the starter wraps it with ContextPropagatingTaskDecorator</code> unless you’ve already done so. Same for ThreadPoolExecutor</code> used in @Async</code> methods.</p>
For Reactor (the hard case), the starter registers a ContextPropagationContext</code> supplier that hooks into Reactor’s Context</code> API. When a trace scope is created, its attributes (trace ID, span ID) are stored in Reactor’s Context</code> automatically. Downstream operators see them.</p>
The trade-off: automatic wrapping assumes you want context propagated everywhere. If you have hot loops doing @Async</code> work millions of times per second, the TaskDecorator</code> overhead (copying the context map) matters. You can disable it for specific thread pools via properties and manage context manually.</p>

#</a>
Extension Points</h2>
ObservationPredicate</strong></p>
Decides whether an observation scope should be recorded. Use it to exclude health checks, readiness probes, and other noisy low-value calls.</p>
@</span>Bean</span></span>
public</span> ObservationPredicate</span> observationPredicate</span>(</span>)</span> {</span></span>
  return</span> (</span>name</span>,</span> context</span>)</span> -></span> {</span></span>
    //</span> Skip /actuator/* paths</span></span>
    if</span> (</span>name</span>.</span>contains</span>(</span>"</span>http.server</span>"</span>)</span> &&</span> </span></span>
        context</span>.</span>getAttribute</span>(</span>"</span>http.url</span>"</span>)</span> !=</span> null</span>)</span> {</span></span>
      String</span> url</span> =</span> (</span>String</span>)</span> context</span>.</span>getAttribute</span>(</span>"</span>http.url</span>"</span>)</span>;</span></span>
      if</span> (</span>url</span>.</span>startsWith</span>(</span>"</span>/actuator</span>"</span>)</span>)</span> return</span> false</span>;</span></span>
    }</span></span>
    return</span> true</span>;</span></span>
  }</span>;</span></span>
}</span></span></code></pre>
ObservationFilter</strong></p>
Mutates attributes on an observation before it’s recorded. Rename attributes, add custom tags, suppress high-cardinality values.</p>
@</span>Bean</span></span>
public</span> ObservationFilter</span> observationFilter</span>(</span>)</span> {</span></span>
  return</span> (</span>context</span>)</span> -></span> {</span></span>
    //</span> Bucket transaction amounts to $1K ranges</span></span>
    if</span> (</span>context</span>.</span>getAttribute</span>(</span>"</span>transaction.amount</span>"</span>)</span> !=</span> null</span>)</span> {</span></span>
      long</span> amount</span> =</span> (</span>Long</span>)</span> context</span>.</span>getAttribute</span>(</span>"</span>transaction.amount</span>"</span>)</span>;</span></span>
      long</span> bucket</span> =</span> (</span>amount </span>/</span> 1000</span>)</span> *</span> 1000</span>;</span></span>
      context</span>.</span>addAttribute</span>(</span>"</span>transaction.amount.bucket</span>"</span>,</span> bucket</span>)</span>;</span></span>
    }</span></span>
  }</span>;</span></span>
}</span></span></code></pre>
ObservationHandler</strong></p>
Custom code that runs when a scope is created, stopped, or encounters an error. Use it to log, send alerts, or update custom metrics.</p>
@</span>Bean</span></span>
public</span> ObservationHandler</span><</span>Observation</span>.</span>Context</span>></span> customHandler</span>(</span>)</span> {</span></span>
  return</span> new</span> ObservationHandler</span><</span>Observation</span>.</span>Context</span>></span>(</span>)</span> {</span></span>
    @</span>Override</span></span>
    public</span> void</span> onStart</span>(</span>Observation</span>.</span>Context</span> context</span>)</span> {</span></span>
      //</span> Custom logic</span></span>
    }</span></span>
    @</span>Override</span></span>
    public</span> void</span> onStop</span>(</span>Observation</span>.</span>Context</span> context</span>)</span> {</span></span>
      //</span> Custom logic</span></span>
    }</span></span>
    @</span>Override</span></span>
    public</span> boolean</span> supportsContext</span>(</span>Observation</span>.</span>Context</span> context</span>)</span> {</span></span>
      return</span> true</span>;</span></span>
    }</span></span>
  }</span>;</span></span>
}</span></span></code></pre>
ObservationRegistryCustomizer</strong></p>
Fine-grained control: add or remove handlers, set sampler strategies, configure exporters.</p>
@</span>Bean</span></span>
public</span> ObservationRegistryCustomizer</span><</span>MeterRegistry</span>></span> registryCustomizer</span>(</span>)</span> {</span></span>
  return</span> (</span>registry</span>)</span> -></span> {</span></span>
    registry</span>.</span>observationConfig</span>(</span>)</span></span>
      .</span>observationHandler</span>(</span>myHandler</span>)</span></span>
      .</span>sampler</span>(</span>Sampler</span>.</span>alwaysSample</span>(</span>)</span>)</span>;</span> //</span> Override default sampler</span></span>
  }</span>;</span></span>
}</span></span></code></pre>
The extension point model:</strong></p>

ObservationPredicate</code> — decides whether to record</li>
ObservationFilter</code> — mutates attributes before recording</li>
ObservationHandler</code> — runs custom code on lifecycle events</li>
ObservationRegistryCustomizer</code> — configures the registry itself</li>
</ul>
Teams implement these as beans and the starter discovers them automatically via @ConditionalOnMissingBean</code>.</p>

#</a>
Reactive & Async Propagation</h2>
This is where teams diverge most. Reactive code (Reactor, Project Loom virtual threads) and traditional @Async</code> need different context propagation strategies.</p>
Reactor (the easy case now, hard historically)</strong></p>
Spring Framework 6.1+ and Spring Boot 3.2+ integrated Micrometer’s ContextPropagationContext</code> directly into Reactor. When you create an observation scope, its attributes are automatically stored in Reactor’s Context</code> and flow through all downstream operators (map, flatMap, switchMap). No manual plumbing.</p>
@</span>Service</span></span>
public</span> class</span> OrderService</span> {</span></span>
  @</span>Autowired</span> private</span> Observation</span> observation</span>;</span></span>
  </span></span>
  public</span> Mono</span><</span>Order</span>></span> createOrder</span>(</span>Order</span> order</span>)</span> {</span></span>
    //</span> Observation scope is created; trace ID flows into Reactor context</span></span>
    return</span> observation</span>.</span>observe</span>(</span>(</span>)</span> -></span></span>
      orderRepository</span>.</span>save</span>(</span>order</span>)</span></span>
        .</span>flatMap</span>(</span>saved </span>-></span> </span></span>
          //</span> Trace ID is visible here</span></span>
          notificationService</span>.</span>notify</span>(</span>saved</span>)</span></span>
        )</span></span>
    )</span>;</span></span>
  }</span></span>
}</span></span></code></pre>
@Async (the complex case)</strong></p>
@Async</code> methods run on a thread pool (TaskExecutor</code>). Without context propagation, the trace ID stays on the caller’s thread. The async work is invisible to tracing.</p>
The starter wraps your TaskExecutor</code> with ContextPropagatingTaskDecorator</code>, which copies the MDC</code> and Spring’s SecurityContext</code> (and, in Spring 6.1+, ContextPropagationContext</code>) to the thread pool thread. The trace ID follows.</p>
But there’s a catch: ForkJoinPool</code> (used by CompletableFuture</code> and parallelStream()</code>) is also a Executor</code>, but Spring doesn’t auto-wrap it. If your service uses CompletableFuture</code> for async work, the starter can’t help—context won’t propagate. You have to either:</p>

Avoid CompletableFuture</code>; use @Async</code> instead.</li>
Manually wrap your CompletableFuture</code> chains with context copy/restore logic (tedious).</li>
Accept that ForkJoinPool</code> work loses trace context (the pragmatic choice for non-critical paths).</li>
</ol>
Real example: a telecom billing system processes CDRs in parallel using parallelStream()</code> on a ForkJoinPool</code>. The main trace covers the CDR ingestion. Each record’s processing (validation, enrichment, ledger update) runs on the ForkJoinPool</code> and loses the trace context. The starter can’t help. Solution: convert to a TaskExecutor</code>-backed @Async</code> loop, accept the sequential overhead, and regain tracing.</p>

#</a>
Data Access Layer Coverage</h2>
JDBC DataSource (the problem)</strong></p>
javax.sql.DataSource</code> is wrapped by Spring with a JdbcObservationFilter</code>, which emits spans for each SQL query. Default behavior captures the SQL and result count. But on a high-throughput service, every unique SQL query becomes a cardinality explosion.</p>
Example: a payments ledger service runs SELECT * FROM ledger WHERE account_id = ?</code> millions of times per day. Each account_id is a different query in the span attributes</em>. You hit your cardinality limits in hours.</p>
The starter lets you filter or rename these attributes:</p>
@</span>Bean</span></span>
public</span> ObservationFilter</span> jdbcFilter</span>(</span>)</span> {</span></span>
  return</span> (</span>context</span>)</span> -></span> {</span></span>
    if</span> (</span>"</span>sql.query</span>"</span>.</span>equals</span>(</span>context</span>.</span>getName</span>(</span>)</span>)</span>)</span> {</span></span>
      //</span> Remove the WHERE clause; just keep the table name</span></span>
      String</span> sql</span> =</span> (</span>String</span>)</span> context</span>.</span>getAttribute</span>(</span>"</span>sql.query</span>"</span>)</span>;</span></span>
      String</span> normalized</span> =</span> sql</span>.</span>replaceAll</span>(</span>"</span>WHERE.*</span>"</span>,</span> "</span>WHERE ...</span>"</span>)</span>.</span>intern</span>(</span>)</span>;</span></span>
      context</span>.</span>addAttribute</span>(</span>"</span>sql.query</span>"</span>,</span> normalized</span>)</span>;</span></span>
    }</span></span>
  }</span>;</span></span>
}</span></span></code></pre>
R2DBC (the solution)</strong></p>
R2DBC (reactive relational database connectivity) ships with first-class Micrometer Observation support. Statements are wrapped automatically, span attributes are pre-normalized, and cardinality is sane.</p>
Trade-off: not all JDBC drivers have R2DBC equivalents (Oracle’s R2DBC support came late). If you’re on PostgreSQL or MySQL, R2DBC is available and observability is better. If you’re on an older proprietary database, you’re stuck with JDBC.</p>
The starter doesn’t force R2DBC, but it strongly recommends it and configures JDBC cardinality filtering to match R2DBC’s behavior.</p>
Data access observability paths:</strong></p>

JDBC Path:</strong> Raw SQL + account_id in spans → cardinality explosion</li>
JDBC + Starter Filter:</strong> Normalized SQL → sane cardinality</li>
R2DBC Path:</strong> Native support, pre-normalized → sane cardinality by default</li>
</ul>

#</a>
Rollout Plan</h2>
You don’t flip a switch and add the starter to 200 services overnight. Context propagation bugs, cardinality explosions, and extension point conflicts will surface only under production load.</p>
Phase 1: Adoption by one team (weeks 1–4)</strong></p>
The platform team dogfoods the starter in one non-critical service (e.g., user preferences microservice). Goals:</p>

Verify context propagation works with their tech stack (Spring Cloud Gateway, Reactor, PostgreSQL).</li>
Validate that Jaeger/Tempo shows correct traces.</li>
Load-test to catch cardinality issues early.</li>
</ul>
Learnings: the preferences team uses CompletableFuture</code> in a recommendations enrichment pipeline. Context is lost. They either switch to @Async</code> or accept the tracing gap. Decision: accept it (it’s low-priority work).</p>
Phase 2: Broader adoption (weeks 5–12)</strong></p>
Roll out to 3–5 services across different teams: payments, ledger, notifications, compliance. Each team configures the starter slightly differently (different ObservationPredicates</code>, different sampler strategies).</p>
Learnings:</p>
The ledger team’s observations on UPDATE account SET balance = balance + ?</code> create one span per unique balance delta. High-cardinality disaster. The starter’s ObservationFilter</code> saves them; they normalize deltas to $100 buckets.</p>
The payments team runs two rate-limiting endpoints side by side (old and new). Both are instrumented. Overlapping @Observed</code> annotations cause spans to be double-counted in Prometheus. Solution: add an ObservationPredicate</code> to suppress the old endpoint’s observations.</p>
The compliance service has audit logging (@Observed</code> on every repository method) and fine-grained tracking (another layer of observability for reporting). Too many observations, Prometheus metrics grow 10x. Solution: introduce a sampler that records 10% of low-cardinality compliance observations and 100% of high-cardinality suspicious transaction flags.</p>
Phase 3: Ecosystem-wide (months 4–6)</strong></p>
Roll out to all services. By this point, the starter has stabilized, and the platform team has published a runbook: “What to do when your observations aren’t showing up”, “How to fix cardinality explosions”, “When to use predicates vs. filters”.</p>
Real example: a major telecom processing 10 million CDRs per day rolls out the starter across their main pipeline (ingestion, validation, enrichment, ledger, billing). Initial cardinality issues: validation step touches 50K equipment IDs per second. They filter observations to suppress equipment-specific tags. Ledger step touches customer service plans (40K plans). They normalize to plan buckets. After 4 weeks of tuning, observability is good, cardinality is predictable.</p>

#</a>
Trade-offs & Failure Modes</h2>
Duplicate observations (silent SLO corruption)</strong></p>
If two teams instrument the same code path, you get two spans for one operation. Prometheus metrics double-count duration and error rates.</p>
Example: the ledger team adds @Observed</code> to TransactionRepository.save()</code>. Later, the core platform team adds the same annotation in a base repository class. Now both fire. Metrics for save duration look good, but they’re double. SLO alerts fire because “p99 duration increased”, but it’s an instrumentation artifact.</p>
Detection: compare span counts in Jaeger to API call counts. If spans > calls, you have duplication.</p>
Fix: the starter includes an optional DuplicateObservationDetector</code> that logs when two observations with the same name are active on the same thread. It’s disabled by default (low-overhead but not free) and can be enabled per service.</p>
#</span> In application.yml</span></span>
s</span>pring.observability.duplicate-detection-enabled</span>:</span> true</span></span></code></pre>
Missing context in async retry queues</strong></p>
A payments service publishes a retry event to Kafka when a ledger update fails. The consumer reads the event, but the trace ID is not in the Kafka header (because the producer didn’t serialize it). The consumer starts a fresh trace. Two disjoint traces where there should be one.</p>
Fix: the starter provides an ObservationContextMessagePropagator</code> bean that automatically serializes trace context into Kafka headers (via Spring Cloud Stream / Spring Kafka).</p>
#</span> Automatic, but only if you use Spring Kafka or Spring Cloud Stream</span></span>
s</span>pring.application.name</span>:</span> p</span>ayments-service</span></span>
#</span> The starter detects Kafka and adds the propagator</span></span></code></pre>
Cardinality explosion from normalized-too-late filters</strong></p>
You define an ObservationFilter</code> that normalizes high-cardinality values. But the span is already created and exported before the filter runs. You’re still emitting millions of unique metric names to Prometheus.</p>
Lesson: use ObservationPredicate</code> (which runs before observation creation) to prevent high-cardinality scopes from being recorded at all. Use ObservationFilter</code> only to tag transformations, not to suppress.</p>
Wrong (ineffective):</p>
@</span>Bean</span></span>
public</span> ObservationFilter</span> wrongApproach</span>(</span>)</span> {</span></span>
  return</span> context </span>-></span> {</span></span>
    if</span> (</span>isTooHighCardinality</span>(</span>context</span>)</span>)</span> {</span></span>
      //</span> Too late! Span already created and emitted</span></span>
      context</span>.</span>addAttribute</span>(</span>"</span>filtered</span>"</span>,</span> true</span>)</span>;</span></span>
    }</span></span>
  }</span>;</span></span>
}</span></span></code></pre>
Right (efficient):</p>
@</span>Bean</span></span>
public</span> ObservationPredicate</span> rightApproach</span>(</span>)</span> {</span></span>
  return</span> (</span>name</span>,</span> context</span>)</span> -></span> {</span></span>
    if</span> (</span>name</span>.</span>equals</span>(</span>"</span>sql.query</span>"</span>)</span> &&</span> isTooHighCardinality</span>(</span>context</span>)</span>)</span> {</span></span>
      return</span> false</span>;</span> //</span> Don't record this observation at all</span></span>
    }</span></span>
    return</span> true</span>;</span></span>
  }</span>;</span></span>
}</span></span></code></pre>
Virtual threads and context leaks</strong></p>
With Project Loom’s virtual threads, you can create millions of threads with negligible overhead. But if context propagation isn’t set up correctly, each virtual thread gets a copy of the trace context, and you lose the parent-child relationship in Jaeger.</p>
Spring 6.1+ handles this automatically via ContextPropagationContext</code> and VirtualThreadTaskExecutor</code>. The starter configures it. But if you use raw Thread.startVirtualThread()</code> without the Spring abstraction, context doesn’t flow.</p>
Mitigation: the starter docs emphasize using Spring’s TaskExecutor</code> abstraction, not raw threads. For teams that ignore this, tracing gaps appear in Jaeger.</p>

#</a>
Cost & Ownership</h2>
Who maintains this?</strong></p>
The platform team. It’s a platform investment, not a framework. Three engineers: one for Micrometer/OpenTelemetry knowledge, one for Spring Boot and context propagation, one for incident response.</p>
How is it distributed?</strong></p>
Publish to your internal Artifactory / Nexus as com.yourcompany.platform:spring-boot-starter-observability:1.0.0</code>. Every service includes it as a compileOnly</code> or implementation</code> dependency.</p>
Versioning: semantic versioning. Minor version bumps when you add new extension points. Patch bumps for bug fixes. When Spring Boot or Micrometer releases a major version, you bump the starter’s major version and validate it across a tier of services before rolling out.</p>
Failure mode: the starter has a bug</strong></p>
A critical bug in context propagation logic is discovered: trace IDs are being dropped in 0.01% of async tasks (enough to break financial reconciliation). You cut a patch release, publish to Artifactory, and all services pick it up in their next deploy.</p>
If you’re using a vendored observability agent, this is the vendor’s problem. With a custom starter, it’s yours. You handle the incident in 2 hours. That’s the ownership cost.</p>
Failure mode: the starter has a breaking change</strong></p>
You upgrade to Micrometer 1.13, which changes the ObservationHandler</code> interface signature. The starter is compatible, but your team’s custom ObservationHandler</code> breaks. You have to backport the custom handler to the new API, or the service can’t deploy.</p>
This is why the starter is versioned carefully. You don’t bump Micrometer versions in the starter without extensive validation and a clear upgrade path for custom handlers.</p>

#</a>
Closing</h2>
If I were to start over, I’d push harder on the separation between “what to observe by default” and “how teams override those defaults”. We learned this too late: a single application.properties</code> file with 47 observability flags is harder to maintain than a few well-designed extension points. Teams are smarter than you think. Give them the levers, set reasonable defaults, and they’ll extend it without breaking things.</p>
Also: observe in production, not in development. Early emphasis on “let’s instrument everything in the test environment” wasted weeks. Real issues surface when you’re processing millions of transactions per day, and that’s when you need to tune. Start with broad observations, filter aggressively, and add precision where it matters.</p>
The other thing: context propagation is harder than metrics. Metrics are fire-and-forget. Context has</em> to flow through async boundaries, and there are always new frameworks (Vert.x, Kotlin coroutines, custom thread pools) that break it. You can’t predict all of them. The starter provides good defaults, but every new technology that touches async gets a post-mortem: “Why didn’t trace context follow?”</p>
Still, building this saved the organization millions in observability tooling costs and weeks of debugging time. A platform team that owns observability is worth the investment.</p>

#</a>
References</h2>

OpenTelemetry Java Automatic Spring Boot Instrumentation</a> — Latest automatic instrumentation guidance from OpenTelemetry, confirms bytecode-injection patterns and OpenTelemetry exporter setup.</li>
Micrometer Observation API</a> — Core API reference for ObservationPredicate</code>, ObservationHandler</code>, ObservationFilter</code>, and registry customization.</li>
Spring Boot Actuator Observability</a> — Spring Boot’s native observability integration, context propagation, and async handler configuration.</li>
Spring Framework 6.1 Context Propagation</a> — Virtual thread and Reactor context propagation design (referenced via OpenTelemetry docs, confirmed in Spring 6.1+ release notes).</li>
R2DBC Observability</a> — R2DBC’s native Micrometer integration and cardinality control compared to JDBC.</li>
</ul>

Solving the Dual-Write Problem with Spring Boot Starter Outbox 2026-07-05T00:00:00+00:00 #</a> The Problem</h2> Publishing messages to a message broker inside a database transaction is unreliable. If the transaction rolls back after publishing, you have a phantom message. If publishing fails after the transaction commits, you have a lost message. Consider this flow in a typical microservice: Your database — source of truth for business state (orders, users, shipments)</li> Your message broker — communication channel to other services (Kafka topics, RabbitMQ exchanges)</li> </ol> When an order is placed, you naturally want to save it to the database AND publish an event to notify other services. But this naive approach has a critical race condition: Broker publish succeeds, then DB transaction rolls back → consumers see an order that never existed (phantom message)</li> DB transaction commits, then broker publish fails → consumers never hear about the order (lost message)</li> Process crashes between the two writes → either phantom or lost message, depending on when</li> </ul> Traditional fixes like two-phase commit, distributed transactions, or ad-hoc retry logic are costly, slow, and still vulnerable to failures. Event sourcing is powerful but requires an architectural commitment that many teams can’t justify. The Transactional Outbox pattern elegantly sidesteps the problem. #</a> Why the Outbox Pattern</h2> The pattern works in three phases: Phase 1 - Atomic Transaction (Synchronous): Save both the business entity AND the outbox event row in the same database transaction. Both writes succeed together or fail together. No partial states. Phase 2 - Relay and Publisher (Asynchronous): A separate scheduler polls the outbox table for PENDING events and publishes them to the message broker. If the broker is down, events remain PENDING in the database. No loss. Phase 3 - Message Broker (Asynchronous): Events are published to topics/exchanges. Downstream consumers subscribe and process them idempotently using the event ID (UUID) as a deduplication key. Key insight: If the database commit succeeds, the outbox row definitely exists. If it fails, neither the business data nor the event row persist. No inconsistency. This creates a strong consistency boundary between the database and the message broker, allowing them to maintain ACID guarantees internally while being eventually consistent externally. #</a> Architectural Benefits</h2> Exactly-once semantics (achieved with consumer deduplication): Combined with consumer idempotency via event ID, you get at-least-once from the pattern plus exactly-once from the consumer.</li> No broker coupling: The service is resilient to broker downtime. Events queue in the database until the broker recovers.</li> Failure isolation: Broker issues don’t cascade to the business transaction.</li> Observability: Failed events are recorded in the database with retry counts and error messages.</li> Partition key preservation: The aggregateId becomes the Kafka partition key, preserving per-entity ordering.</li> Separation of concerns: Database writes are decoupled from message broker concerns via the relay mechanism.</li> Operational simplicity: No distributed transactions or two-phase commit needed.</li> </ul> #</a> The Outbox Table</h2> All events live in a single table, outbox_events</code>: CREATE TABLE outbox_events ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), aggregate_type VARCHAR(100) NOT NULL, aggregate_id VARCHAR(255) NOT NULL, event_type VARCHAR(100) NOT NULL, payload TEXT NOT NULL, status VARCHAR(20) NOT NULL DEFAULT 'PENDING', retry_count INTEGER NOT NULL DEFAULT 0, error_message TEXT, created_at TIMESTAMPTZ NOT NULL DEFAULT now(), processed_at TIMESTAMPTZ ); CREATE INDEX idx_outbox_status_created ON outbox_events(status, created_at);</code></pre> Column meanings: id</code> - Deduplication key for consumers (required for idempotent processing)</li> aggregate_type</code> / aggregate_id</code> - Business routing (which entity owns this event)</li> event_type</code> - Event classification (maps to Kafka topic or RabbitMQ routing key)</li> payload</code> - JSON serialization of the domain event</li> status</code> - PENDING (awaiting relay), PROCESSED (successfully delivered), FAILED (exhausted retries)</li> retry_count</code> - Number of attempts made</li> error_message</code> - Last failure reason for debugging</li> created_at</code> - When the event was persisted (DB commit time)</li> processed_at</code> - When the relay successfully delivered it</li> </ul> #</a> Data Flow Diagram</h2> The flow moves through three logical layers: graph TD subgraph Service["Service Layer (Business Logic)"] OS["OrderService.createOrder()"] TXN["@Transactional Boundary"] DBW["Save Order to DB"] OUTBOX["Publish Event to Outbox"] end subgraph Database["PostgreSQL Database"] ORDERS["orders table"] OUTBOXTBL["outbox_events table (status=PENDING)"] end subgraph Relay["OutboxEventRelay (@Scheduled)"] POLL["Poll: SELECT WHERE status=PENDING"] SEND["Broker Adapter.send()"] UPDATE["UPDATE status=PROCESSED"] end subgraph Broker["Kafka or RabbitMQ"] TOPIC["Topic/Exchange"] end subgraph Consumer["Downstream Services"] CS["Consumer Service"] DEDUP["Deduplicate by event.id"] PROCESS["Process Event"] end OS --> TXN TXN --> DBW DBW --> ORDERS TXN --> OUTBOX OUTBOX --> OUTBOXTBL OUTBOXTBL -->|COMMIT| RELAY RELAY --> POLL POLL -->|Every 5s| SEND SEND --> TOPIC TOPIC --> CS CS --> DEDUP DEDUP --> PROCESS</code></pre> #</a> Failure Modes and Recovery</h2> Failure</th> Handled By</th></tr></thead> DB commit succeeds, broker publish fails</td> Relay retries on next cycle (automatic recovery)</td></tr> DB transaction rolls back</td> Event row never inserted; no phantom message</td></tr> Relay crashes mid-cycle</td> Next cycle picks up PENDING events (idempotent)</td></tr> Broker is down for hours</td> Events remain PENDING in DB; relay publishes when broker recovers</td></tr> Event permanently malformed</td> Marked FAILED after max-retries; operator reviews error_message</td></tr> Duplicate delivery</td> Consumer must deduplicate using event id (UUID)</td></tr> </tbody></table> Monitor the outbox table for events stuck in PENDING state (age > 5× relay interval) or accumulating in FAILED state. These are early warning signs of broker issues or payload corruption. #</a> Implementation Guide</h2> Prerequisites: Spring Boot 3.x or 4.x</li> Java 17+</li> Spring Data JPA with PostgreSQL (or compatible database with UUID support)</li> Apache Kafka or RabbitMQ</li> Gradle or Maven</li> </ul> Add the starter to build.gradle.kts: dependencies { implementation("io.github.saumilp.starters:spring-boot-starter-outbox:1.0.0") implementation("org.springframework.kafka:spring-kafka:3.3.0") // or: // implementation("org.springframework.boot:spring-boot-starter-amqp:3.2.0") }</code></pre> Step 1: Create the Outbox Table Run the SQL DDL above against your PostgreSQL database. The table should be created during initial schema setup or via Flyway/Liquibase migrations. Step 2: Configure the Starter In application.yml</code>: spring: outbox: enabled: true relay-interval-ms: 5000 max-retries: 3 backoff-multiplier: 2.0 kafka: bootstrap-servers: localhost:9092 topic-prefix: outbox.</code></pre> Step 3: Implement Your Service @Service public class OrderService { private final OrderRepository orderRepository; private final OutboxPublisher outboxPublisher; @Transactional public Order createOrder(CreateOrderRequest req) { Order order = orderRepository.save(new Order(req)); outboxPublisher.publish( new OrderCreatedEvent( order.getId(), order.getCustomerId(), order.getTotal() ) ); return order; } }</code></pre> The @Transactional</code> annotation ensures both the order AND the outbox event are persisted atomically. Step 4: Consuming Events Consumers must deduplicate by event ID to handle at-least-once delivery: @Service public class OrderEventConsumer { private final OrderEventRepository eventRepository; @KafkaListener(topics = "outbox.ORDER_CREATED") public void handleOrderCreated(OrderCreatedEvent event) { // Check if already processed if (eventRepository.existsByEventId(event.getId())) { return; } // Process the event processOrder(event); // Record that we processed it eventRepository.save(new ProcessedEvent(event.getId())); } }</code></pre> Consumer Idempotency Patterns: In-memory deduplication (low-traffic): Store event IDs in a HashSet. Simple but lost on restart. </li> Database deduplication (recommended): Store processed event IDs in a database table with unique constraint. Survives restarts. </li> Idempotent business operations (best): Make the operation itself idempotent. Use event ID as a key. If the operation runs twice, it’s safe. </li> Database constraints (clever): Use UNIQUE constraint on (event_id, business_key). The second insert fails safely. </li> </ol> #</a> Infrastructure and Deployment</h2> Docker Compose for local development: version: '3.8' services: postgres: image: postgres:16 environment: POSTGRES_DB: outbox_demo POSTGRES_PASSWORD: postgres ports: - "5432:5432" volumes: - postgres_data:/var/lib/postgresql/data kafka: image: confluentinc/cp-kafka:7.6.0 depends_on: - zookeeper ports: - "9092:9092" environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true" zookeeper: image: confluentinc/cp-zookeeper:7.6.0 ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 volumes: postgres_data:</code></pre> Testing the flow: Start services: docker-compose up</code></li> Create an order: curl -X POST http://localhost:8080/orders -H "Content-Type: application/json" -d '{"customerId":"cust-123", "amount":99.99}'</code></li> Check outbox table: SELECT * FROM outbox_events;</code></li> Verify Kafka topic has the event: kafka-console-consumer.sh --topic outbox.ORDER_CREATED --bootstrap-servers localhost:9092</code></li> Verify consumer processed it: SELECT * FROM processed_events;</code></li> </ol> CI/CD Pipeline (GitHub Actions): name: Test Outbox Starter on: [push, pull_request] jobs: test: runs-on: ubuntu-latest services: postgres: image: postgres:16 env: POSTGRES_PASSWORD: postgres options: >- --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 steps: - uses: actions/checkout@v3 - uses: actions/setup-java@v3 with: java-version: 21 distribution: temurin - run: ./gradlew test --info</code></pre> #</a> Key Takeaways</h2> For Architects: The Outbox pattern is NOT a substitute for event sourcing. Use it for reliable event publishing in transaction-based services. For temporal queries, event replay, or audit trails, event sourcing is the better choice. For Developers: Remember: the pattern provides at-least-once delivery, so consumers MUST be idempotent. Use the event ID as your deduplication key. Test both success and failure paths (relay down, broker down, transaction rollback). For DevOps: Monitor the outbox table cardinality and relay latency. Set up alerts for FAILED events accumulating. The relay is stateless, so you can run multiple instances for redundancy. Best Practices: Always use event IDs for consumer deduplication</li> Make business operations idempotent, not just event deduplication</li> Set reasonable max-retries (usually 3-5)</li> Use exponential backoff for retries</li> Log failed events with full error messages</li> Partition by aggregateId for ordering guarantees</li> Never delete from outbox table; archive to cold storage after 30 days</li> </ul> Kafka vs RabbitMQ Trade-offs: Aspect</th> Kafka</th> RabbitMQ</th></tr></thead> Ordering</td> Per-partition (use aggregateId)</td> Per-queue (declare exclusive queues)</td></tr> At-least-once</td> Built-in with consumer groups</td> Requires manual ACKs</td></tr> Retention</td> Days/weeks (configurable)</td> Until consumed</td></tr> Scaling</td> Horizontal (partitions)</td> Vertical plus clustering</td></tr> Complexity</td> Moderate (Zookeeper/KRaft)</td> Simple setup</td></tr> </tbody></table> For the Outbox pattern, Kafka’s partition-based ordering is ideal. For RabbitMQ, use topic exchanges with routing keys bound to dedicated queues. #</a> References and Further Reading</h2> Transactional Outbox Pattern: https://microservices.io/patterns/data/transactional-outbox.html</li> Spring Boot Starter Outbox: https://github.com/saumilpatel/spring-boot-starter-outbox</li> Event-Driven Architecture: https://www.oreilly.com/library/view/building-event-driven-microservices/</li> Kafka at-least-once semantics: https://kafka.apache.org/documentation/#consumerconfigs_enable.auto.commit</li> Spring Data JPA: https://spring.io/projects/spring-data-jpa</li> PostgreSQL UUID type: https://www.postgresql.org/docs/current/datatype-uuid.html</li> </ul> #</a> Troubleshooting</h2> Q: Events are stuck in PENDING status A: Check if the relay is running (logs should show “Polling outbox…”). Verify broker connectivity. Check error_message column for details. Q: “Transaction already rolled back” error when calling publish() A: OutboxPublisher must be called within a @Transactional method. The transaction must be active and not already marked for rollback. Q: Duplicate messages in the broker A: The pattern guarantees at-least-once. Consumers MUST deduplicate by event ID. This is not a bug; it’s the contract. Q: Relay is taking too long (> relay-interval-ms) A: Increase relay-interval-ms or add database indexes. If outbox_events table is huge, partition it by created_at or status. Q: How do I handle event schema changes? A: Version your event payloads (add a “version” field). Consumers should support multiple versions. Use JSON Schema or Protocol Buffers for validation. Q: Can I use this with RabbitMQ? A: Yes. Configure rabbitmq section instead of kafka in application.yml. The pattern is broker-agnostic. Colophon: how this site is built 2026-06-28T00:00:00+00:00 This site is a static, dependency-light technical publication. Generator: Zola</a> — a single fast Rust binary, no Node build pipeline.</li> Templates: custom Tera</a> partials and macros.</li> Styles: a hand-written SCSS design system compiled by Zola, driven by CSS custom properties (light + dark).</li> Search: Pagefind</a> — a fully static search index generated after the build.</li> Interactions: a few tiny vanilla-JS modules (theme toggle, copy-code, mobile nav, search, TOC) — no framework.</li> Hosting: GitHub Pages, built and deployed by GitHub Actions.</li> </ul> No client-side framework, no CDN dependency, no external analytics. Java Review MCP Server 2025-02-01T00:00:00+00:00 The problem</h2> AI-assisted code review requires understanding both the syntax and the broader system design context. Generic code review tools miss enterprise-specific concerns: transaction boundaries, observability instrumentation, context propagation patterns, and domain-driven design principles. Integrating code review into a developer’s workflow without context switching is difficult. Approach</h2> Built an MCP (Model Context Protocol) server that exposes a Java project’s structure and analysis capabilities to Claude, enabling context-aware code reviews. The server parses the codebase, builds an AST-based understanding of packages, classes, and dependencies, then provides Claude with structured access to perform deep reviews. Key capabilities: Project structure analysis and dependency graph visualization</li> Class and method-level code analysis with architectural context</li> Transaction boundary detection and verification</li> Observability instrumentation pattern recognition</li> Security vulnerability pattern matching</li> Architecture constraint validation</li> </ul> Trade-offs</h2> Full project analysis over isolated review: Slower startup but enables holistic review feedback. Caches analysis to reduce latency on repeated reviews.</li> Claude as reviewer over specialized linters: More context-aware and human-friendly feedback, but requires language model access and costs per review.</li> MCP protocol over custom API: Ensures interoperability with Claude and other tools, but adds protocol overhead.</li> </ul> Outcome</h2> Seamless integration of AI-assisted code review into Claude workflows. Teams get sophisticated architectural review feedback without leaving their editor or IDE. Particularly valuable for enforcing enterprise patterns across distributed teams and mentoring junior engineers on system design principles. Dev machine setup for Flutter App development 2024-11-16T00:00:00+00:00 Setting up Flutter</a> on macOS can be staightforward process when using Homebrew</a>, a popular package manager for macOS. Flutter, a powerful open-source UI software development toolkit by Google, enables developers to build natively compiled mobile, web, and desktop applications from a single codebase. In this guide, we’ll walk you through the necessary steps to install and configure Flutter on macOS system using Homebrew. Additionally, we will cover the installation of essential tools like Android Studio and Xcode, update Ruby, and set up CocoaPods to ensure a smooth development experience. Rust Learning Lab 2024-06-15T00:00:00+00:00 The problem</h2> Learning Rust requires understanding both the language syntax and the systems programming concepts it enables. Most tutorials focus on syntax, while real-world systems programming involves memory safety, concurrency patterns, and performance optimization. There’s a gap between “Hello, World!” and building production-grade systems in Rust. Approach</h2> Created a comprehensive learning project that progresses from fundamentals to advanced systems programming. The lab covers ownership and borrowing through practical examples, explores concurrency patterns with real-world scenarios, and includes deep dives into performance optimization. Key areas: Core concepts: Ownership, borrowing, lifetimes, and the type system with runnable examples</li> Systems programming: Memory management, unsafe code patterns, and performance profiling</li> Concurrency: Threading, async/await, channels, and lock-free patterns</li> Algorithms & data structures: Implementations with trade-off analysis</li> Performance: Benchmarking, profiling, and optimization techniques</li> </ul> Trade-offs</h2> Breadth over depth: Covers many topics at intermediate level rather than mastering one deeply. Designed as a learning progression, not a reference.</li> Examples over libraries: Most implementations are from scratch to teach concepts, not production-ready code.</li> Educational comments over minimal code: Heavy documentation and explanations to aid learning, which would be excessive in production code.</li> </ul> Outcome</h2> A self-directed learning resource for engineers transitioning to Rust or deepening their systems programming knowledge. Particularly valuable for those coming from higher-level languages who need to understand memory safety and performance characteristics. Serves as both a learning guide and a reference for Rust patterns in real systems. Learning to fly with Rust 2024-04-01T00:00:00+00:00 Recently I updated my blog-site with Zola</a>, and then I started thinking to host Rust application on cloud providers. There are multiple CSPs ( Cloud Service Providers ) enabling Rust enthusiasts Developers (also known as “Rustaceans”) to host web-apps on cloud. Zola quickstart 2024-03-19T00:00:00+00:00 This post is one of the first one after a long time, I am evaluating Zola</code></a> as replacement of Jekyll, and thought of creating a blog for step-by-step site building. Whether you like Rust and SSG or you do not know any of these, if you wish to create a website or a blog, Zola</code> is made for you! We are going to discover here the basic concepts.… Mastodon Toot Client 2024-02-01T00:00:00+00:00 The problem</h2> Mastodon’s official apps don’t support scheduling or file-based content. Automating posts for a personal account — say, a weekly architecture tip from a text file — required either a Python script with fragile state management or a paid third-party service. Approach</h2> Built a stateless Rust CLI: reads a text file of toots (one per line), posts the next unposted one to the Mastodon API, and marks it done by moving the line to a done</code> file. Runs as a GitHub Actions cron job — no server required, credentials stored as Actions secrets. Trade-offs</h2> Stateless file-append over a database: Simpler deployment and easier to audit. The “done” file doubles as a history log.</li> Rust over Python: Compile-time guarantees matter when the binary runs unattended. The async Tokio runtime handles the HTTP call without blocking.</li> </ul> Outcome</h2> A zero-dependency automation that posts on a schedule without any hosted infrastructure. The approach generalises to any API-driven content automation with similar stateless patterns. Spring Boot Starters 2024-01-15T00:00:00+00:00 The problem</h2> Building microservices requires solving the same problems repeatedly: atomic event publishing, context propagation across async boundaries, cardinality control in observability pipelines, and idempotent message consumption. Each team implements these differently, leading to fragmented traces, inconsistent retry logic, and high-cardinality metrics explosions. Approach</h2> Created a suite of focused Spring Boot starters that encode opinions about how to solve these problems consistently across a microservice mesh. Each starter is independently deployable and composable, allowing teams to adopt one, several, or all without vendor lock-in. Key starters: spring-boot-starter-outbox: Transactional outbox pattern for reliable event publishing with automatic relay and retry logic</li> spring-boot-starter-observability: Enforced context propagation, cardinality controls, and extension points for custom instrumentation</li> spring-boot-starter-idempotency: Consumer-side deduplication with pluggable backends (in-memory, Redis, database)</li> </ul> Trade-offs</h2> Auto-configuration over manual setup: Faster onboarding but requires trust in the starter’s opinions. Overrideable via @ConditionalOnMissingBean</code>.</li> Outbox relay over broker transactions: Avoids distributed transaction complexity but introduces a scheduled component that must be monitored.</li> Platform-level observability over per-team instrumentation: Ensures consistency but loses flexibility for teams with unique requirements.</li> </ul> Outcome</h2> A production-proven library reducing time-to-reliability for event-driven systems. Eliminates boilerplate around event publishing, context propagation, and idempotent processing. Used across fintech and telecom microservice architectures handling millions of events per day. draw.io Architecture Libraries 2024-01-01T00:00:00+00:00 The problem</h2> Architecture diagrams in most teams suffer from inconsistency — each engineer invents their own notation, colour scheme, and level of abstraction. Reviews spend more time decoding the diagram than discussing the decision. Approach</h2> Built a curated set of draw.io shape libraries covering C4 model levels (Context, Container, Component), AWS service icons, and a custom notation layer for domain events and async flows. Libraries are XML files that can be imported directly into draw.io desktop or app.diagrams.net with one click. Trade-offs</h2> Chose draw.io over Mermaid/PlantUML: Draw.io diagrams are visual-first and accessible to non-engineers. Mermaid is better for version control but harder to layout precisely.</li> Static XML over dynamic generation: Keeps the library dependency-free and usable offline. The trade-off is that updating icons requires editing XML by hand.</li> </ul> Outcome</h2> The libraries are used in personal architecture documentation and have been shared with colleagues for review. The C4 layer in particular cuts diagram preparation time significantly by providing pre-built component shapes with consistent colour coding. Enterprise Spring Patterns 2023-09-01T00:00:00+00:00 The problem</h2> Spring Boot makes the happy path easy, but enterprise systems have edge cases: transactional outbox patterns, idempotent consumers, domain event publishing, and layered service boundaries. These topics are scattered across blog posts with inconsistent advice. Approach</h2> Built a reference application demonstrating enterprise patterns with real infrastructure: PostgreSQL for persistence, Kafka for event streaming, and Redis for idempotency keys. Each pattern is isolated in its own package with an integration test that uses Testcontainers. Trade-offs</h2> Testcontainers over H2: H2 in-memory DB hides PostgreSQL-specific behaviour. Testcontainers is slower to start but catches real issues.</li> Outbox pattern over direct Kafka publish: Ensures at-least-once delivery without distributed transactions.</li> </ul> Outcome</h2> A self-contained reference for backend engineers migrating from monolith Spring MVC to event-driven architectures. Covers the gap between “Hello World” Spring Boot tutorials and production-grade system design. Java Design Patterns 2023-06-01T00:00:00+00:00 The problem</h2> Design patterns are often taught with UML diagrams and abstract descriptions. Real engineers need runnable code they can step through, modify, and connect to frameworks they already use (Spring, CDI, etc.). Approach</h2> Each pattern gets its own Maven module: a minimal implementation, a Spring Boot integration where it makes sense, and JUnit 5 tests that assert the expected behaviour. Patterns are documented with a short “when to use / when not to” note rather than a textbook definition. Trade-offs</h2> Maven multi-module over Gradle: Heavier to set up but more familiar to enterprise Java developers who are the primary audience.</li> Spring integration optional: Not every pattern maps naturally to Spring — forced integration would produce misleading examples.</li> </ul> Outcome</h2> A living reference that maps GoF patterns to real Java 21 idioms (records, sealed classes, switch expressions) and framework-level equivalents in Spring Boot 3. gh-stats 2023-03-01T00:00:00+00:00 The problem</h2> GitHub contribution graphs and repository statistics are useful for portfolios and dashboards, but embedding them requires either heavy JavaScript libraries or manual updates. Serverless functions are perfect for this — lightweight, scalable, and stateless — but most implementations over-engineer the solution with unnecessary dependencies. Approach</h2> Built a minimal serverless function that queries the GitHub API and generates clean SVG visualizations. The service is stateless, dependency-light, and optimized for fast response times. Each visualization is generated on-demand, eliminating cache invalidation headaches. Key capabilities: Contribution graph: Visual timeline of GitHub contributions</li> Repository stats: Stars, forks, language distribution</li> User stats: Public profile metrics and activity summaries</li> SVG generation: No external rendering services — pure SVG generation</li> Caching headers: Appropriate cache directives for dashboard embedding</li> </ul> Trade-offs</h2> SVG over canvas/images: SVG is lightweight and infinitely scalable, but rendering is limited to basic shapes and text. No photorealistic charts.</li> On-demand over pre-computed: Function calls add latency, but eliminate stale data and infrastructure complexity.</li> Minimal dependencies: Reduces bundle size and attack surface, but requires more custom logic than using charting libraries.</li> </ul> Outcome</h2> A lightweight tool for developers who want GitHub stats on their websites, portfolios, or dashboards without heavy client-side libraries. Particularly useful for status pages, portfolio sites, and internal dashboards where fresh data matters but heavy infrastructure doesn’t. Automatic ID generation in Apache Solr 2015-02-16T00:00:00+00:00 I have been working on Apache Solr for last few months, and have been recieving requirements to speed up query process. As part of the investigation, i found out as retrieved documents’ unique id generation contributes query processing.And hence i have decided to add this post.… XML Schema Design Patterns 2015-02-09T00:00:00+00:00 This might be one of the old topic - specially when lot of developers are working with RESTful Services, but i would like to point out couple of pros and cons. As most of the developers know, XML schema is very powerful, but they lack object-orientation, and are intended to capture a data model rather than an object model. In last one and half decades, there has been few tools available to architects and developers to be able to make use off some of the object orientation principles - like polymorphism for flexibility. But ultimately the whole point of the services must be general enough that it can communicate across languages. Hence i have attempted to put together some information around some of the very well known Design patterns.… Curried Function in Java 2015-01-31T00:00:00+00:00 Currying is a technique of transforming a function with multiple arguments into a function with just single argument. The single argument is the value of the first argument from the original function and it returns another single argument function. This in turn would take the second original argument and itself return another single argument function. This kind of chaining continues over the arguments of the original. The last in the chain will have access to all the arguments and so can do whatever it needs to do.… Quick Apache Solr Setup 2015-01-31T00:00:00+00:00 It has been long time since I started with setup/configuration related blog post. This post provides steps for setting up developer machine for Apache Solr</a>. Apache Solr is an opensource search engine written in Java, from the Apache Lucene project - with many powerful features</a>. Closures in Java 2015-01-23T00:00:00+00:00 Working on last few projects, i’ve realized that use of closures can come really handy in day-to-day work. Can you imagine writing loops after loops to extract or collect properties from list of custom objects? It can be tiring and painful. And hence, I have decided to add this post - mainly to point out couple of situations where introduction to closures can really ease this pain.…

Machine Mind

Custom Spring Boot Observability Starter: When Every Team's Instrumentation Becomes Your Bottleneck

Solving the Dual-Write Problem with Spring Boot Starter Outbox

Colophon: how this site is built

Java Review MCP Server

Dev machine setup for Flutter App development

Rust Learning Lab

Learning to fly with Rust

Zola quickstart

Mastodon Toot Client

The problem</h2> Mastodon’s official apps don’t support scheduling or file-based content. Automating posts for a personal account — say, a weekly architecture tip from a text file — required either a Python script with fragile state management or a paid third-party service.</p>

Approach</h2> Built a stateless Rust CLI: reads a text file of toots (one per line), posts the next unposted one to the Mastodon API, and marks it done by moving the line to a done</code> file. Runs as a GitHub Actions cron job — no server required, credentials stored as Actions secrets.</p>

Outcome</h2> A zero-dependency automation that posts on a schedule without any hosted infrastructure. The approach generalises to any API-driven content automation with similar stateless patterns.</p>

Spring Boot Starters

Outcome</h2> A production-proven library reducing time-to-reliability for event-driven systems. Eliminates boilerplate around event publishing, context propagation, and idempotent processing. Used across fintech and telecom microservice architectures handling millions of events per day.</p>

draw.io Architecture Libraries

The problem</h2> Architecture diagrams in most teams suffer from inconsistency — each engineer invents their own notation, colour scheme, and level of abstraction. Reviews spend more time decoding the diagram than discussing the decision.</p>

Outcome</h2> The libraries are used in personal architecture documentation and have been shared with colleagues for review. The C4 layer in particular cuts diagram preparation time significantly by providing pre-built component shapes with consistent colour coding.</p>

Enterprise Spring Patterns

The problem</h2> Spring Boot makes the happy path easy, but enterprise systems have edge cases: transactional outbox patterns, idempotent consumers, domain event publishing, and layered service boundaries. These topics are scattered across blog posts with inconsistent advice.</p>

Approach</h2> Built a reference application demonstrating enterprise patterns with real infrastructure: PostgreSQL for persistence, Kafka for event streaming, and Redis for idempotency keys. Each pattern is isolated in its own package with an integration test that uses Testcontainers.</p>

Outcome</h2> A self-contained reference for backend engineers migrating from monolith Spring MVC to event-driven architectures. Covers the gap between “Hello World” Spring Boot tutorials and production-grade system design.</p>

Java Design Patterns

The problem</h2> Design patterns are often taught with UML diagrams and abstract descriptions. Real engineers need runnable code they can step through, modify, and connect to frameworks they already use (Spring, CDI, etc.).</p>

Approach</h2> Each pattern gets its own Maven module: a minimal implementation, a Spring Boot integration where it makes sense, and JUnit 5 tests that assert the expected behaviour. Patterns are documented with a short “when to use / when not to” note rather than a textbook definition.</p>

Outcome</h2> A living reference that maps GoF patterns to real Java 21 idioms (records, sealed classes, switch expressions) and framework-level equivalents in Spring Boot 3.</p>

gh-stats

Outcome</h2> A lightweight tool for developers who want GitHub stats on their websites, portfolios, or dashboards without heavy client-side libraries. Particularly useful for status pages, portfolio sites, and internal dashboards where fresh data matters but heavy infrastructure doesn’t.</p>

Automatic ID generation in Apache Solr

XML Schema Design Patterns

Curried Function in Java

Quick Apache Solr Setup

Closures in Java

The problem</h2>
Mastodon’s official apps don’t support scheduling or file-based content. Automating posts for a personal account — say, a weekly architecture tip from a text file — required either a Python script with fragile state management or a paid third-party service.</p>

Approach</h2>
Built a stateless Rust CLI: reads a text file of toots (one per line), posts the next unposted one to the Mastodon API, and marks it done by moving the line to a `done</code> file. Runs as a GitHub Actions cron job — no server required, credentials stored as Actions secrets.</p>`

Outcome</h2>
A zero-dependency automation that posts on a schedule without any hosted infrastructure. The approach generalises to any API-driven content automation with similar stateless patterns.</p>

Outcome</h2>
A production-proven library reducing time-to-reliability for event-driven systems. Eliminates boilerplate around event publishing, context propagation, and idempotent processing. Used across fintech and telecom microservice architectures handling millions of events per day.</p>

The problem</h2>
Architecture diagrams in most teams suffer from inconsistency — each engineer invents their own notation, colour scheme, and level of abstraction. Reviews spend more time decoding the diagram than discussing the decision.</p>

Outcome</h2>
The libraries are used in personal architecture documentation and have been shared with colleagues for review. The C4 layer in particular cuts diagram preparation time significantly by providing pre-built component shapes with consistent colour coding.</p>

The problem</h2>
Spring Boot makes the happy path easy, but enterprise systems have edge cases: transactional outbox patterns, idempotent consumers, domain event publishing, and layered service boundaries. These topics are scattered across blog posts with inconsistent advice.</p>

Approach</h2>
Built a reference application demonstrating enterprise patterns with real infrastructure: PostgreSQL for persistence, Kafka for event streaming, and Redis for idempotency keys. Each pattern is isolated in its own package with an integration test that uses Testcontainers.</p>

Outcome</h2>
A self-contained reference for backend engineers migrating from monolith Spring MVC to event-driven architectures. Covers the gap between “Hello World” Spring Boot tutorials and production-grade system design.</p>

The problem</h2>
Design patterns are often taught with UML diagrams and abstract descriptions. Real engineers need runnable code they can step through, modify, and connect to frameworks they already use (Spring, CDI, etc.).</p>

Approach</h2>
Each pattern gets its own Maven module: a minimal implementation, a Spring Boot integration where it makes sense, and JUnit 5 tests that assert the expected behaviour. Patterns are documented with a short “when to use / when not to” note rather than a textbook definition.</p>

Outcome</h2>
A living reference that maps GoF patterns to real Java 21 idioms (records, sealed classes, switch expressions) and framework-level equivalents in Spring Boot 3.</p>

Outcome</h2>
A lightweight tool for developers who want GitHub stats on their websites, portfolios, or dashboards without heavy client-side libraries. Particularly useful for status pages, portfolio sites, and internal dashboards where fresh data matters but heavy infrastructure doesn’t.</p>