The Spring documentation explicitly states the preference order for connection pooling:

1. HikariCP: Preferred for its performance and concurrency. If available, it is always chosen.

2. Tomcat Pooling: The second choice if HikariCP is unavailable.

3. Commons DBCP2: The third alternative.

4. Oracle UCP: Used as a last resort.

Let’s dive into why HikariCP is considered the "gold standard" and how we can optimize it for live environments.

Why Use a Connection Pool (DBCP) at All?

In raw JDBC, connecting to a database is an expensive operation. Every user request would require:

1. Loading the DB driver.

2. Establishing a TCP/IP connection (the infamous 3-way handshake).

3. Authentication (sending ID/PW) and creating a DB session.

4. Returning the connection object to the client.

Doing this for every request is extremely inefficient. A Connection Pool pre-allocates these connections so the application can simply "borrow" and "return" them, bypassing the heavy handshake and initialization overhead.

Performance Analysis

The benchmark results from the HikariCP team show a staggering difference:

• Connection Cycle (ops/ms): HikariCP handles ~50,000 operations per millisecond, while its closest competitor (Vibur) manages only about 1/10th of that.

• Statement Cycle (ops/ms): The gap remains just as wide, proving HikariCP's dominance in executing SQL statements.

The Secret Sauce: How HikariCP Achieves This

HikariCP goes "down the rabbit hole" with low-level optimizations:

1. Bytecode-level Engineering: The team studied JIT (Just-In-Time) compiler assembly output to ensure critical routines stay under the "Inline Threshold." By making methods inlineable, they eliminate the overhead of method calls.

2. CPU Cache Optimization: They optimized the code to fit within L1/L2 caches. By minimizing instructions, they ensure tasks complete within the OS scheduler's time slice, avoiding the performance hit of a "cache miss" when a thread is rescheduled to a different core.

3. Custom Collections (FastList): They replaced ArrayList with a custom FastList.

   • Removed Range Checks: It skips index validation (e.g., checking if the index exceeds array size) to save cycles.

   • Optimized Scans: It performs removal scans from head to tail, specifically optimized for the way connection pools access statements.

MySQL Performance Optimization Options

The HikariCP team suggests several data-source-properties to maximize MySQL throughput:

1. PreparedStatement Caching

• cachePrepStmts=true: Enables client-side caching of PreparedStatement objects.

• prepStmtCacheSize=250: Number of statements to cache per connection. (Default: 25, Recommended: 250–500).

• prepStmtCacheSqlLimit=2048: Maximum length of a SQL string to cache. Essential for long ORM-generated queries.

2. Server-Side & Protocol Optimization

• useServerPrepStmts=true: Instead of sending full SQL strings, it sends a template to the server and only passes parameters thereafter. This reduces network traffic and allows the DB to reuse execution plans.

• rewriteBatchedStatements=true: Optimizes bulk INSERT/UPDATE operations.

• useLocalSessionState=true: Tracks session state locally to avoid unnecessary round-trips to the server.

3. Metadata & Stats

• elideSetAutoCommits=true: Eliminates redundant setAutoCommit calls.

• maintainTimeStats=false: Disables internal timing metrics to reduce overhead.

  • Note: While this boosts performance, it makes troubleshooting harder as you lose metrics like connection acquisition time.

Recommended Configuration for Production

Database Level (Reference)

Ensure your DB server is configured to handle the pool size.

MySQL:

max_connections = 1000
innodb_buffer_pool_size = 4G
wait_timeout = 28800

Spring Boot application.yml

Here is a production-ready configuration focused on performance and stability.

spring:
  datasource:
    url: jdbc:mysql://db-url:3306/dbname?rewriteBatchedStatements=true&characterEncoding=UTF-8&serverTimezone=Asia/Seoul&useSSL=true
    username: ${DB_USERNAME}
    password: ${DB_PASSWORD}
    driver-class-name: com.mysql.cj.jdbc.Driver
    hikari:
      pool-name: HikariCP-Primary
      # Sizing: (core_count * 2) + effective_spindle_count
      maximum-pool-size: 10  
      minimum-idle: 5
      idle-timeout: 300000     # 5 mins
      connection-timeout: 5000 # 5 secs (Fail fast)
      max-lifetime: 1200000    # 20 mins (Must be shorter than DB wait_timeout)
      auto-commit: true
      data-source-properties:
        cachePrepStmts: true
        prepStmtCacheSize: 250
        prepStmtCacheSqlLimit: 2048
        useServerPrepStmts: true
        useLocalSessionState: true
        rewriteBatchedStatements: true
        cacheResultSetMetadata: true
        cacheServerConfiguration: true
        elideSetAutoCommits: true
        maintainTimeStats: false

  jpa:
    database-platform: org.hibernate.dialect.MySQLDialect
    hibernate:
      ddl-auto: validate
    properties:
      hibernate:
        show_sql: false
        format_sql: false
        jdbc:
          batch_size: 100
          order_inserts: true
          order_updates: true
        query:
          in_clause_parameter_padding: true

Conclusion

HikariCP isn't just "fast"; it's engineered at the bytecode level to respect CPU architecture. By combining its lean design with the proper MySQL driver properties, you can significantly reduce latency in your backend services.

As a web backend developer, you quickly realize how frequently ObjectMapper is used. In a typical Controller, the Jackson library's ObjectMapper handles the serialization of JSON requests into our desired object types. It is also common practice for developers to inject ObjectMapper to handle data when interacting with Redis.

To be honest, I didn't find the code above strange at first. In fact, based on my experience, I thought writing it this way was the only way to perfectly control the frequent exceptions thrown by ObjectMapper. This assumption likely stemmed from not looking deeply enough into the internal libraries that Spring relies on.

How ObjectMapper Operates in a Controller

1. Identify Content-Type: When an HTTP request arrives, the server checks the Content-Type.

2. Trigger Jackson: If it is application/json, Jackson’s ObjectMapper is invoked.

3. Deserialization: ObjectMapper converts the JSON string into an instance of the specified class.

Serialization > - The process of converting an object in memory into a format that can be stored or transmitted.

• Converting Java/Kotlin objects into JSON strings or byte streams.

Deserialization > - The process of converting stored or transmitted data back into an object that can be used in memory.

• Converting JSON strings back into Java/Kotlin objects.

A quick question: Does ObjectMapper also work when communicating with a Database?

• No. ObjectMapper is primarily responsible for converting JSON objects during HTTP communication.

• When communicating with a DB, JPA maps objects (Entities) based on table metadata, while MyBatis maps SQL results to objects.

So far, we know ObjectMapper deserializes JSON strings into objects. However, data actually arrives at the server as a byte stream, not a raw JSON string. Let’s look at where that transformation happens.

1. Client sends JSON: {"name": "Kim", "age": 25}

2. Network Transmission:

   • The HTTP request is converted into a byte stream.

   • Content-Type: application/json is included in the header.

3. Spring Server:

   • Byte stream → Converted to JSON string.

   • ObjectMapper converts the JSON string → Kotlin/Java object.

Who Converts Byte Streams to JSON?

In Spring MVC, the HttpMessageConverter is responsible for converting the HTTP request byte stream into a JSON string.

1. Arrival: The HTTP request byte stream arrives.

2. Read: The byte stream is read via ServletInputStream.

3. Process: MappingJackson2HttpMessageConverter takes over.

   • It uses ObjectMapper internally.

   • It converts the byte stream into a string using InputStreamReader.

4. Convert: ObjectMapper converts the JSON string into an object.

Looking at Spring's default configuration, we can see where converters are added in the WebMvc configuration classes.

// Inside WebMvcConfigurationSupport
if (jackson2XmlPresent) {
    Jackson2ObjectMapperBuilder builder = Jackson2ObjectMapperBuilder.xml();
    if (this.applicationContext != null) {
        builder.applicationContext(this.applicationContext);
    }
    messageConverters.add(new MappingJackson2XmlHttpMessageConverter(builder.build()));
    // ...
}

If you examine MappingJackson2HttpMessageConverter, it inherits from AbstractJackson2HttpMessageConverter. Inside that class, you can see the part where it receives the byte stream through an InputStream object.

private Object readJavaType(JavaType javaType, HttpInputMessage inputMessage) throws IOException {
    // ...
    try {
        InputStream inputStream = StreamUtils.nonClosing(inputMessage.getBody());
        // ...

End-to-End Flow Summary

• Initial HTTP Processing: Request arrives → Tomcat Connector assigns a thread → Request parsed into HttpServletRequest.

• FilterChain: Passes through DelegateFilter → SecurityFilter → etc.

• DispatcherServlet: doDispatch() is called → Find Handler via HandlerMapping → Execute via HandlerAdapter.

• @RequestBody Processing: Read body via ServletInputStream → HttpMessageConverter (Byte Stream → JSON → Object).

• @ResponseBody Processing: Return value processed by HttpMessageConverter (Object → JSON → Byte Stream) → Response written via ServletOutputStream.

Moving Beyond Try-Catch: ObjectMapper Configurations

Let's analyze the exceptions handled in the original code:

1. JsonProcessingException: The root exception for Jackson. It covers all issues during JSON parsing or generation (syntax errors, incomplete strings). Since these are often "human errors" from the client side, we might still need some level of control here.

2. JsonMappingException: A subclass of JsonProcessingException for specific mapping issues (type mismatch, missing fields).

   • Realization: My original code caught JsonProcessingException first. Since it's the parent, JsonMappingException would never be caught in its own block. I should have analyzed the library hierarchy more carefully.

Recommended ObjectMapper Configurations

Instead of messy try-catches, we can configure ObjectMapper to handle many common issues gracefully.

1. Basic & Deserialization Settings

objectMapper.apply {
    setSerializationInclusion(JsonInclude.Include.NON_NULL) // Exclude nulls
    configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) // Ignore unknown fields
    configure(DeserializationFeature.READ_UNKNOWN_ENUM_VALUES_AS_NULL, true) // Unknown Enum as null
    registerModule(JavaTimeModule()) // Support Java 8 Date/Time
}

2. Custom Wrapper for Clean Code

Since we cannot completely eliminate JsonProcessingException, I recommend using a Wrapper class:

@Component
@Slf4j
class JsonConverter(private val objectMapper: ObjectMapper) {
    fun <T> fromJson(json: String, type: Class<T>): Optional<T> {
        return try {
            Optional.ofNullable(objectMapper.readValue(json, type))
        } catch (e: JsonProcessingException) {
            log.error("JSON Conversion Failed: {}", e.getMessage())
            Optional.empty()
        }
    }
}

Supplemental: Issues with MappingJackson2HttpMessageConverter

I’d like to share an issue discussed in my dev community regarding MappingJackson2HttpMessageConverter.

The Problem: When communicating with an external partner API using WebClient or RestClient, an error occurred stating the request body was empty. Interestingly, it worked fine with OpenFeign or when sending data as a raw String.

The Cause: Chunked Transfer Encoding. When you pass an Object directly to the request body, MappingJackson2HttpMessageConverter triggers. In Spring 6.1+, to optimize memory, RestTemplate and RestClient no longer buffer the request body by default. Consequently, the Content-Length header is not set, and the data is sent using Transfer-Encoding: chunked.

If the external partner's server does not support chunked encoding, it fails.

Solutions:

1. Override getContentLength in MappingJackson2HttpMessageConverter.

2. Wrap the ClientHttpRequestFactory with BufferingClientHttpRequestFactory to force buffering (and thus set Content-Length).

3. Send the data as a String (which uses StringHttpMessageConverter that provides a Content-Length).

This change was documented in the Spring Framework 6.1 Release Notes to optimize memory usage. It’s a crucial reminder that keeping up with release notes is just as important as writing clean code!

• This is known as the Native Thread Implementation or the 1:1 Threading Model.

Thread Pool in Spring Boot

• By default, Spring Boot creates a thread pool when using an embedded Tomcat server.

• The default configuration allows for a maximum of 200 threads.

• Each incoming request is handled by a worker thread from this pool.

Q1. If I spin up a container using a Docker image, does that container get assigned processes and threads from the host OS?

1. Relationship Between Containers and Processes

• A Docker container runs as an isolated group of processes on the host OS.

• It leverages Linux namespaces and cgroups to isolate processes and limit resource usage.

• In reality, it shares the host OS kernel.

2. In the Case of Spring Boot Applications

• The JVM running inside the container is also a process on the host OS.

• JVM threads are still mapped to the host OS's native threads.

• However, they are subject to the container's resource constraints (CPU, Memory, etc.).

3. Verification Commands

Bash

# Check container processes from the host OS
docker top [container-id]

# Check processes inside the container
docker exec [container-id] ps -ef

# Check specific threads
docker exec [container-id] ps -eLf

1. Container Structure

• Each container includes its own application, libraries, and runtime.

• Containers remain isolated from one another.

2. Namespace Isolation

• PID Namespace: Isolates Process IDs.

  • A process may have PID 1 inside the container but appear as PID 1234 on the host system.

• Network Namespace: Isolates the network stack.

  • Provides each container with an independent network stack (interfaces, IP addresses, routing tables, port numbers, iptables rules).

  • When a container is created, Docker generates a veth (virtual ethernet) pair.

  • One end resides in the host's network namespace, while the other (eth0) resides in the container's.

  • These interfaces act like a pipe to forward traffic.

• Port Mapping

  • -p 8080:80

• Mount Namespace: Isolates filesystem mount points.

• User Namespace: Isolates User and Group IDs.

3. Resource Control (cgroups)

• Manages CPU, Memory, and I/O usage.

4. Kernel Sharing

• All containers share the same host OS kernel and access system resources through it.

Q2. In a Kotlin Spring Boot environment, are the threads in the Dispatchers pool used by Coroutines also OS threads?

1. Dispatcher Thread Pools

• Dispatchers.Default: Uses a thread pool proportional to the number of CPU cores (Minimum 2, Maximum: CPU cores + 1).

• Dispatchers.IO: Uses a shared thread pool (defaults to up to 64 threads).

• All of these are indeed native OS threads.

2. Coroutines vs. Threads

• While Coroutines are called "lightweight threads," they are actually units of work executed on top of threads.

• Multiple coroutines can run on a single thread (M:N mapping).

• Switching between coroutines is significantly "lighter" than switching between threads.

Q3. When a Coroutine Scope is executed, is a specific thread type assigned? And is switching between coroutines really cheaper than thread context switching?

1. Coroutine Scope and Dispatchers

Kotlin

// Determine which thread pool to use via the Dispatcher
CoroutineScope(Dispatchers.Default).launch {
    // All coroutines in this scope use the Default thread pool

    // Switching between coroutines (e.g., during suspension) is extremely lightweight
    val result1 = async { heavyComputation() }
    val result2 = async { anotherComputation() }
}

2. Context Switching Cost Comparison

• Thread Context Switch: Occurs at the OS level (Expensive).

  • Requires saving/loading CPU register states: Program Counter (PC), Stack Pointer (SP), General-purpose registers, and Status registers.

Thread Switching Process:

1. Thread A is running -> Save Thread A's register values to memory (in the PCB).

2. Load Thread B's register values from memory.

3. Start Thread B execution.

What is a PCB (Process Control Block)?

A metadata structure maintained by the OS to manage each process.

• Management Info: PID, Status, Priority, PC, CPU Registers, Scheduling info.

• Memory Info: Allocation, Page/Segment tables.

• File/IO Info: Open files, File descriptors, I/O devices.

• Memory Map Switching: Switching the virtual memory address space.

  • TLB Flushing: Increases page table walks as the translation cache becomes invalid.

  • Increased Cache Misses/Page Faults: Since the new process needs different data, memory access latency increases.

• CPU Cache Invalidation: Data in the L1/L2/L3 caches may no longer be valid for the new thread, leading to cache misses.

• Coroutine Switching:

  • Occurs within the same thread.

  • Only saves execution state and stack information.

  • Extremely lightweight with minimal memory access.

Kotlin

// Conceptual structure of Coroutine Suspension (Continuation)
fun example(continuation: ExampleContinuation) {
    when(continuation.label) {
        0 -> {
            continuation.x = 10
            continuation.label = 1 
            delay(1000, continuation)
        }
        1 -> {
            val x = continuation.x // Restore state
            println(x)
            continuation.label = 2
            delay(2000, continuation)
        }
        2 -> {
            val x = continuation.x
            println(x + 1)
        }
    }
}

Q4. If I set the Spring Boot Max Thread Pool to 200, are Async and Coroutine threads created separately?

Yes, they use independent thread pools:

1. Tomcat Thread Pool: server.tomcat.threads.max=200 (For HTTP requests).

2. Spring @Async Pool: Configured separately via ThreadPoolTaskExecutor.

3. Coroutine Dispatcher Pool: Default and IO dispatchers maintain their own pools.

Total Thread Count = Tomcat Threads + @Async Threads + Coroutine Threads.

In a Docker environment, you must be cautious as the sum of these pools can lead to heavy resource contention.

Q5. If I run a JAR in Docker with Max Threads at 200, 2 Async threads, and default Coroutine settings, how are Coroutine threads calculated?

• Tomcat: Max 200.

• Async: 2.

• Dispatchers.Default: min(Core Count * 2, 128).

• Dispatchers.IO: Max 64.

For an 8-core system:

• $200 (Tomcat) + 2 (Async) + 16 (Default) + 64 (IO) = 282$ potential threads.

  Note: Threads are created/destroyed dynamically based on demand; they aren't all allocated at startup.

Q6. What happens to resources if I spin up a second identical container on the same host?

• Container 1: Up to 282 threads.

• Container 2: Up to 282 threads.

• Total: Up to 564 threads.

Critical Point: Both containers share the same host CPU and memory. Without limits, they will compete for resources, leading to resource contention and performance degradation. It is best practice to define limits:

YAML

# docker-compose.yml example
services:
  app1:
    cpus: '4'
    pids_limit: 150
  app2:
    cpus: '4'
    pids_limit: 150

Q7. Maximize Performance: Large Server with Many Containers vs. Multiple Small Servers?

Verdict: It depends. Complementary services should be grouped, while high-load/mission-critical services should be isolated.

Q8. In K8s (1 Master, 3 Nodes), does scaling out mean adding more containers to the same server?

Scaling out in Kubernetes follows two paths:

1. Sufficient Node Resources: The K8s Scheduler places new Pods (containers) on existing nodes. Multiple containers will run on one server.

2. Insufficient Node Resources: Pods enter a Pending state. You must add physical/virtual nodes. In cloud environments, Cluster Autoscaler can automate this.

Pro-tip: Always define resources.requests and limits to make scaling predictable.

When I first joined my previous company, I found that despite being a solution provider, there wasn't a properly established core solution. We had one product delivered to a client, but the codebase—likely due to a rushed timeline—lacked consistent conventions. I immediately felt that this project desperately needed verification for its tangled logic and various technical debts.

As we began building a new version of the solution, I proposed the introduction of test codes to my colleagues. Although they had little experience with automated testing, and I hadn't written extensive test code in a production environment myself, we all agreed on its necessity and decided to move forward.

Feeling the Necessity of TDD

The journey wasn't exactly smooth, but following the footsteps of many senior developers online, I gradually integrated test codes into the project. I added tests to existing business logic and established rules for different testing scopes. At the time, I thought I was writing well-organized, rule-compliant test code.

However, at some point, I felt that something was missing. Because I was writing tests after the business logic was already finished, the tests couldn't influence the design itself. In some cases, the tests ended up merely validating poorly designed or flawed logic, making them feel less than half-effective.

That was when TDD (Test-Driven Development) caught my eye. TDD seemed like the perfect approach to improve both design capabilities and requirement verification. Although it felt like a daunting concept at first, I began learning it through formal training. This post is a record of those trials and errors.

Feedback Summary (Before the 3rd Session)

1. Use hasSize() when validating the size of a Collection.

2. Parameters in @ParameterizedTest support automatic type conversion.

3. Replace Magic Numbers and Magic Strings with constants.

4. Do not abbreviate variable names.

5. Minimize the use of getters and setters.

6. Consider the specific nuance of variable names (e.g., check vs. validate vs. verify).

7. If something is difficult to test (like random values), push the responsibility to a higher-level object.

8. Instead of using a getter, send a message to the object (Ask, don't tell).

9. Keep indentation levels to a maximum of one.

10. Limit classes to a maximum of 2-3 instance variables.

11. Wrap primitive values in classes (Value Objects).

12. Utilize First-Class Collections.

13. Follow the "Object Calisthenics" principles.

Object Calisthenics

Rather than viewing these as rigid laws, I treat them as guidelines that increase the probability of producing high-quality code.

1. Only one level of indentation per method.

2. Don't use the else keyword.

3. Wrap all primitives and strings.

4. Use only one dot per line (e.g., object.method(), avoiding deep chaining).

5. Don't abbreviate.

6. Keep all entities small.

7. No classes with more than three instance variables.

8. Use First-Class Collections.

9. No getters, setters, or properties (where possible).

I discovered a bug where the dates in the user activity stream tab—which summarizes the last three days of activity—were stuck at the exact time the Docker container was started.

Troubleshooting

Attempt 1: Syncing Container Time with Local Server Time

Initially, I tried to synchronize the container's localtime with the host server's localtime. When the issue occurred, I ran the date command on the server, and it correctly displayed KST. However, the time inside the container was still being output in UTC. My Docker Compose configuration was set to mount /etc/localtime:/etc/localtime.

Attempt 2: Syncing via Timezone File

After some research, I found feedback suggesting that syncing via /etc/timezone is often more reliable than using /etc/localtime. I applied this change immediately, but the container output remained in UTC.

Attempt 3: Installing tzdata in the Dockerfile

To ensure the container could handle time dynamically, I modified the Dockerfile to install the tzdata package. I configured it to copy the zoneinfo of the desired timezone to the container's /etc/localtime and overwrite /etc/timezone with the specific TZ value.

Dockerfile

ENV TZ=Asia/Seoul
RUN apk add --no-cache tzdata && \
    cp /usr/share/zoneinfo/$TZ /etc/localtime && \
    echo $TZ > /etc/timezone

After starting the container with this new image, the internal system time finally matched the server time.

Re-evaluating the Root Cause

Despite fixing the system time, the API was still returning a fixed timestamp. I kept wondering, "How can LocalDateTime, which is supposed to fetch the system time, be a constant value?"

The fact that even the nanoseconds were identical was the smoking gun. It suddenly dawned on me: when defining a DateUtil class for global use, I had declared the date variables as static final.

Feeling a bit foolish, I removed the final keywords and tried registering the class as a Spring @Component (Bean). But of course, the value was still fixed—the instance variable held the value initialized when the singleton bean was first created. It was a lapse in judgment during debugging. After some self-reflection, I refactored the code to either use a standard class or initialize the time at the exact point of execution.

After running all test cases and deploying to the dev server, I confirmed the API works perfectly.

Retrospective

I'm documenting this to remind myself to be more deliberate during development. Following the recent login issue, I really need to break the habit of reflexively registering everything as a Bean without considering its lifecycle.

This is the architecture of the solution we previously deployed to our clients. Most of our clients were universities (occasionally graduate schools, but the requirements were largely similar). These clients generally fell into three categories: those who outsourced server management to a third party, those with an in-house IT team managing the servers, and those with no dedicated IT team or physical server infrastructure at all.

Despite these three categories, they all shared a common constraint: an environment where scaling out servers at will was impossible, primarily due to budget limitations. Given these constraints, our initial delivery strategy was to request the highest-specification server the university could provide and deploy all services onto that single instance. Consequently, all business logic was concentrated within a single monolithic backend server.

Initially, this didn't pose significant issues. Since the solution was typically managed by the university's IT team or relevant departments after delivery, we didn't have much exposure to the operational challenges. However, the friction in this architecture began to surface when a client signed a larger contract that included a full year of operational support for their freshman class.

Requirements

As the project included a commitment to develop additional requirements, the university requested features like QR check-ins for mid-semester events (festivals, faculty consultations, etc.). The challenge arose from the nature of an LMS (Learning Management System) within a university setting: students can engage in coursework at any time.

Therefore, the server had to be constantly available to handle attendance data. We realized that shutting down the server to deploy new features could potentially compromise the data integrity and reliability of attendance records.

After some deliberation, we opted for a manual Blue/Green deployment strategy. We would update half of the running containers first, followed by the remaining half. While this method didn't cause any major incidents—aside from being incredibly tedious and nerve-wracking every time—it allowed us to navigate the semester successfully.

Once the semester concluded and the final grades were delivered, I began to reconsider the architecture for future deployments.

Decoupling Mission-Critical Functions

Based on the insights gained during operations, I decided to isolate the functions that required constant uptime. These primarily included the attendance module and the logic for aggregating assignment and quiz scores. However, since the assignment and quiz logic was subject to frequent changes, I decided not to group it with the attendance module.

Attendance data required zero-tolerance for error. Even if a discrepancy occurred, we needed a system that allowed for rapid feedback and correction via the database where all records were stored.

I further decoupled the servers related to attendance. Since the attendance server needed to be constantly available when requesting data, I designed separate servers to collect video viewing data and Zoom session information.

Lessons from Decoupling

To be clear, decoupling isn't purely beneficial. It adds complexity to areas that were previously easier to develop within a single server. Our decision wasn't driven by a blind desire to follow the MSA (Microservices Architecture) hype; it was a pragmatic choice to reduce the operational burden for future projects. Since we rarely use more than two servers, managing an increasing number of containers could have led to overhead, especially since Kubernetes was overkill for our scale.

However, the conclusion so far is one of great satisfaction. While there are downsides—such as having to fetch data via external API communication—the overall service feels significantly more robust and stable. In the next post, I will review the performance and speed aspects of this change.

Hi, I’m a backend engineer currently working on building and maintaining LMS (Learning Management System) solutions for universities. Due to some "lucky" (?) timing with previous team members departing shortly after I joined, I found myself leading the current projects. This post is a self-reflection—and a bit of a self-reprimand—to ensure I never repeat the same mistakes again.

To give you some context, our team’s tech stack includes:

• Language/Framework: Java 17, Spring Boot 3.x

• Persistence/Query: JPA, QueryDSL, MySQL, PostgreSQL

• Infrastructure/Middleware: Redis, Docker, Nginx, Ubuntu 20.04/22.04

The Incident: A Collision of State

The issue arose during the process of passing classroom information to Canvas LMS via SAML authentication. Specifically, the problem occurred when trying to include metadata about the specific classroom a user intended to access within the SAML payload.

The Initial (Flawed) Approach: My initial thought was: "When a user requests the SAML login URL, let's capture the target classroom info, store it in an object, and then use that data to authenticate and redirect the user."

However, I hit a roadblock. There was no straightforward way to retrieve that stored information when Canvas sent the callback request back to our server. Since Canvas didn't even provide identifying information about which user was making the request at that specific stage, I couldn't simply persist it in the database and query it later.

Under pressure, I made a decision I now deeply regret—a decision born from not yet having "felt" the dangers of stateful Singletons in my bones: I decided to store these user-specific details in a global variable (static/class-level field) within a Singleton bean.

The Symptom: It Works on My Machine

Predictably, the logic worked perfectly during local development. Since I was the only one testing, there were no concurrent requests to expose the horror of global variables. This was also a failure of our QA process; we didn't adequately simulate high-concurrency scenarios.

The nightmare finally manifested during a live client demonstration. As multiple stakeholders logged in simultaneously, they began seeing other people's names and accessing classrooms belonging to different users. It was a catastrophic "identity swap" scenario that I’d rather forget.

The Fix: From Dirty Patches to Proper State Management

Attempt 1 (The Naive Fix): In a panic, my first thought was to keep the global variable but immediately nullify/initialize it after the redirection. This didn't solve the problem; it only reduced the error frequency. If another request hit the server in the millisecond before the variable was cleared, the same collision occurred. Concurrency is not something you can "race" against with manual overrides.

Attempt 2 (The Robust Solution): The final and correct approach was to leverage Client-side State. I chose to store the encrypted classroom metadata in a Cookie. When the flow redirected to the point where Canvas needed the data, the server could retrieve it directly from the user's browser request. This effectively decoupled the state from the server's memory and tied it to the individual user's session. This completely resolved the identity-swapping issue.

Lessons Learned

They say experience is a hard teacher because she gives the test first and the lesson afterward. While this was a "good" experience in the sense that I will now be obsessively cautious about state management and Singleton design, I realize that with just a bit more deep thinking, I could have avoided such a fundamental architectural flaw.

If you’ve experienced something similar, you have my deepest sympathies. If you haven't—let this be a warning: Be extremely vigilant when handling state within Singleton objects. In a multi-threaded environment, global variables are a ticking time bomb.