Added a connection pool (#11)

L

docs/README.md

@@ -24,6 +24,7 @@ Welcome to the sqlgen documentation. This guide provides detailed information ab
 - [sqlgen::update](update.md) - How to update data in a table
 
 - [Transactions](transactions.md) - How to use transactions for atomic operations
+- [Connection Pool](connection_pool.md) - How to manage database connections efficiently
 
 ## Data Types and Validation
 

docs/connection_pool.md

@@ -0,0 +1,243 @@
+# Connection Pool
+
+sqlgen provides a thread-safe connection pool implementation that efficiently manages database connections. The pool automatically handles connection lifecycle, ensures thread safety, and provides RAII-based session management.
+
+## Configuration
+
+The connection pool can be configured using `ConnectionPoolConfig`:
+
+```cpp
+using namespace sqlgen;
+
+ConnectionPoolConfig config{
+    .size = 4,                    // Number of connections in the pool
+    .num_attempts = 10,          // Number of retry attempts when acquiring a connection
+    .wait_time_in_seconds = 1    // Wait time between retry attempts
+};
+
+// Create a pool with the specified configuration
+const auto pool = make_connection_pool<postgres::Connection>(
+    config,
+    postgres::Credentials{
+        .user = "postgres",
+        .password = "password",
+        .host = "localhost",
+        .dbname = "postgres"
+    }
+);
+```
+
+### Configuration Parameters
+
+- `size`: The number of connections to maintain in the pool
+- `num_attempts`: Maximum number of attempts to acquire a connection before giving up
+- `wait_time_in_seconds`: Time to wait between retry attempts when no connections are available
+
+## Basic Usage
+
+### Creating a Connection Pool
+
+Create a connection pool with a specified number of connections:
+
+```cpp
+using namespace sqlgen;
+
+const auto pool = make_connection_pool<postgres::Connection>(
+    config,  // ConnectionPoolConfig
+    credentials // Variables necessary to create the connections
+);
+
+if (!pool) {
+    // Handle error
+    std::cerr << "Failed to create pool: " << pool.error() << std::endl;
+    return;
+}
+```
+
+### Acquiring Sessions
+
+Sessions are acquired from the pool using the `session()` function:
+
+```cpp
+using namespace sqlgen;
+
+// Acquire a session from the pool
+const auto session_result = session(pool);
+
+if (!session_result) {
+    // Handle error (e.g., no available connections)
+    std::cerr << "Failed to acquire session: " << session_result.error() << std::endl;
+    return;
+}
+
+// Use the session
+const auto& sess = session_result.value();
+```
+
+### Chaining Operations
+
+Sessions can be used to chain database operations:
+
+```cpp
+using namespace sqlgen;
+
+const auto result = session(pool)
+    .and_then(drop<Person> | if_exists)
+    .and_then(write(std::ref(people)))
+    .and_then(read<std::vector<Person>>);
+
+if (!result) {
+    // Handle error
+    std::cerr << "Operation failed: " << result.error() << std::endl;
+    return;
+}
+
+const auto& people = result.value();
+```
+
+## Thread Safety
+
+The connection pool is designed to be thread-safe:
+
+- Each connection is protected by an atomic flag
+- Sessions are automatically released when they go out of scope
+- The pool uses atomic operations for connection management
+- Multiple threads can safely acquire and release sessions
+
+Example of thread-safe usage with monadic style:
+
+```cpp
+using namespace sqlgen;
+
+// Create pool and handle error case
+const auto pool = make_connection_pool<postgres::Connection>(4, credentials)
+    .or_else([](const auto& err) {
+        std::cerr << "Failed to create pool: " << err << std::endl;
+        return error(err);
+    });
+
+std::vector<std::thread> threads;
+for (int i = 0; i < 4; ++i) {
+    threads.emplace_back([&pool]() {
+        session(pool)
+            .and_then(update<Person>("age"_c.set(46) | where("id"_c == i)));
+    });
+}
+
+for (auto& thread : threads) {
+    thread.join();
+}
+```
+
+## Session Management
+
+Sessions are managed through RAII (Resource Acquisition Is Initialization) and support monadic operations:
+
+- Sessions automatically release their connections when destroyed
+- Connections are returned to the pool when sessions go out of scope
+- No explicit connection release is required
+
+```cpp
+using namespace sqlgen;
+
+// Using monadic style for session management with exec
+session(pool)
+    .and_then(exec(begin_transaction))
+    .and_then(exec(update<Person>("age"_c.set(46))))
+    .and_then(exec(commit))
+    .or_else([](const auto& err) {
+        // Handle any errors in the chain
+        std::cerr << "Session operation failed: " << err << std::endl;
+        return error(err);
+    });
+
+```
+
+## Pool Statistics
+
+The connection pool provides methods to monitor its state:
+
+```cpp
+using namespace sqlgen;
+
+const auto pool = make_connection_pool<postgres::Connection>(config, credentials);
+
+// Get total number of connections
+const size_t total_connections = pool.value().size();  // Returns 4
+
+// Get number of available connections
+const size_t available_connections = pool.value().available();  // Returns 4 initially
+```
+
+## Connection Acquisition
+
+The pool implements a retry mechanism when acquiring connections:
+
+- If no connection is available, the pool will retry up to `num_attempts` times
+- Between each attempt, it waits for `wait_time_in_seconds`
+- If all attempts fail, an error is returned
+- This helps handle temporary connection unavailability
+
+Example of handling connection acquisition with retries:
+
+```cpp
+using namespace sqlgen;
+
+// Configure pool with retry behavior
+ConnectionPoolConfig config{
+    .size = 4,
+    .num_attempts = 5,
+    .wait_time_in_seconds = 2
+};
+
+const auto pool = make_connection_pool<postgres::Connection>(config, credentials);
+
+// Acquire a session - will retry if no connections are available
+const auto session_result = session(pool)
+    .or_else([](const auto& err) {
+        // Handle connection acquisition failure after all retries
+        std::cerr << "Failed to acquire connection after retries: " << err << std::endl;
+        return error(err);
+    });
+```
+
+## Best Practices
+
+1. **Pool Size**: Choose an appropriate pool size based on your application's needs:
+   - Too small: May cause connection wait times
+   - Too large: May waste resources
+   - Rule of thumb: Start with (2 * number of CPU cores)
+
+2. **Retry Configuration**: Configure retry behavior based on your use case:
+   - For high-availability systems: Use more retry attempts with shorter wait times
+   - For resource-constrained systems: Use fewer retries with longer wait times
+   - Consider your application's latency requirements when setting wait times
+
+3. **Session Lifetime**: Keep sessions as short as possible:
+   ```cpp
+   // Good: Session is released immediately after use
+   session(pool).and_then(execute_query).value();
+
+   // Bad: Session is held longer than necessary
+   const auto sess = session(pool).value();
+   // ... other operations ...
+   sess.execute_query();
+   ```
+
+4. **Transaction Management**: Use transactions within sessions for atomic operations:
+   ```cpp
+   session(pool)
+       .and_then(begin_transaction)
+       .and_then(update<Person>("age"_c.set(46)))
+       .and_then(commit)
+       .value();
+   ```
+
+## Notes
+
+- The connection pool is template-based and works with any sqlgen connection type
+- Sessions are move-only types (cannot be copied)
+- The pool automatically cleans up connections when destroyed
+- All operations return `Result` types for error handling
+- The pool is designed to be efficient and minimize contention
+- Connection acquisition is non-blocking (returns error if no connections available) 

include/sqlgen.hpp

@@ -1,6 +1,7 @@
 #ifndef SQLGEN_HPP_
 #define SQLGEN_HPP_
 
+#include "sqlgen/ConnectionPool.hpp"
 #include "sqlgen/Flatten.hpp"
 #include "sqlgen/Iterator.hpp"
 #include "sqlgen/IteratorBase.hpp"
@@ -10,6 +11,7 @@
 #include "sqlgen/Range.hpp"
 #include "sqlgen/Ref.hpp"
 #include "sqlgen/Result.hpp"
+#include "sqlgen/Session.hpp"
 #include "sqlgen/Timestamp.hpp"
 #include "sqlgen/Varchar.hpp"
 #include "sqlgen/begin_transaction.hpp"

include/sqlgen/ConnectionPool.hpp

@@ -0,0 +1,108 @@
+#ifndef SQLGEN_CONNECTIONPOOL_HPP_
+#define SQLGEN_CONNECTIONPOOL_HPP_
+
+#include <atomic>
+#include <chrono>
+#include <memory>
+#include <numeric>
+#include <thread>
+#include <utility>
+#include <vector>
+
+#include "Ref.hpp"
+#include "Result.hpp"
+#include "Session.hpp"
+
+namespace sqlgen {
+
+struct ConnectionPoolConfig {
+  size_t size = 4;
+  size_t num_attempts = 10;
+  size_t wait_time_in_seconds = 1;
+};
+
+template <class Connection>
+class ConnectionPool {
+  using ConnPtr = Ref<Connection>;
+
+ public:
+  template <class... Args>
+  ConnectionPool(const ConnectionPoolConfig& _config, const Args&... _args) {
+    conns_->reserve(_config.size);
+    for (size_t i = 0; i < _config.size; ++i) {
+      auto conn = Ref<Connection>::make(_args...);
+      auto flag = Ref<std::atomic_flag>::make();
+      flag->clear();
+      conns_->emplace_back(std::make_pair(std::move(conn), std::move(flag)));
+    }
+  }
+
+  template <class... Args>
+  static Result<ConnectionPool> make(const ConnectionPoolConfig& _config,
+                                     const Args&... _args) noexcept {
+    try {
+      return ConnectionPool(_config, _args...);
+    } catch (std::exception& e) {
+      return error(e.what());
+    }
+  }
+
+  ~ConnectionPool() = default;
+
+  /// Acquire a session from the pool. Returns an error if no connections are
+  /// available after several attempts.
+  Result<Ref<Session<Connection>>> acquire() noexcept {
+    for (size_t att = 0; att < config_.num_attempts; ++att) {
+      if (att != 0) {
+        std::this_thread::sleep_for(
+            std::chrono::seconds(config_.wait_time_in_seconds));
+      }
+      for (auto& [conn, flag] : *conns_) {
+        if (!flag->test_and_set()) {
+          return Ref<Session<Connection>>::make(conn, flag);
+        }
+      }
+    }
+    return error("No available connections in the pool.");
+  }
+
+  /// Get the current number of available connections
+  size_t available() const {
+    return std::accumulate(conns_->begin(), conns_->end(), 0,
+                           [](const auto _count, const auto& _p) {
+                             return _p.second->test() ? _count : _count + 1;
+                           });
+  }
+
+  /// Get the total number of connections in the pool
+  size_t size() const { return conns_->size(); }
+
+ private:
+  /// The configuration for the connection pool.
+  ConnectionPoolConfig config_;
+
+  /// The underlying connection objects.
+  Ref<std::vector<std::pair<ConnPtr, Ref<std::atomic_flag>>>> conns_;
+};
+
+template <class Connection, class... Args>
+Result<ConnectionPool<Connection>> make_connection_pool(
+    const ConnectionPoolConfig& _config, const Args&... _args) noexcept {
+  return ConnectionPool<Connection>::make(_config, _args...);
+}
+
+template <class Connection>
+Result<Ref<Session<Connection>>> session(
+    ConnectionPool<Connection> _pool) noexcept {
+  return _pool.acquire();
+}
+
+template <class Connection>
+Result<Ref<Session<Connection>>> session(
+    Result<ConnectionPool<Connection>> _res) noexcept {
+  return _res.and_then([](auto _pool) { return session(_pool); });
+}
+
+}  // namespace sqlgen
+
+#endif

include/sqlgen/Session.hpp

@@ -0,0 +1,97 @@
+#ifndef SQLGEN_SESSION_HPP_
+#define SQLGEN_SESSION_HPP_
+
+#include <atomic>
+#include <memory>
+#include <optional>
+#include <vector>
+
+#include "IteratorBase.hpp"
+#include "Ref.hpp"
+#include "dynamic/Insert.hpp"
+#include "dynamic/SelectFrom.hpp"
+#include "dynamic/Statement.hpp"
+#include "dynamic/Write.hpp"
+
+namespace sqlgen {
+
+template <class Connection>
+class Session {
+ public:
+  using ConnPtr = Ref<Connection>;
+
+  Session(const Ref<Connection>& _conn, const Ref<std::atomic_flag>& _flag)
+      : conn_(_conn), flag_(_flag.ptr()) {}
+
+  Session(const Session<Connection>& _other) = delete;
+
+  Session(Session<Connection>&& _other)
+      : conn_(std::move(_other.conn_)), flag_(_other.flag_) {
+    _other.flag_.reset();
+  }
+
+  ~Session() {
+    if (flag_) {
+      flag_->clear();
+    }
+  }
+
+  Result<Nothing> begin_transaction() { return conn_->begin_transaction(); }
+
+  Result<Nothing> commit() { return conn_->commit(); }
+
+  Result<Nothing> execute(const std::string& _sql) {
+    return conn_->execute(_sql);
+  }
+
+  Result<Nothing> insert(
+      const dynamic::Insert& _stmt,
+      const std::vector<std::vector<std::optional<std::string>>>& _data) {
+    return conn_->insert(_stmt, _data);
+  }
+
+  Session& operator=(const Session& _other) = delete;
+
+  Session& operator=(Session&& _other) noexcept {
+    if (this == &_other) {
+      return *this;
+    }
+    conn_ = std::move(_other.conn_);
+    flag_ = _other.flag_;
+    _other.flag_.reset();
+    return *this;
+  }
+
+  Result<Ref<IteratorBase>> read(const dynamic::SelectFrom& _query) {
+    return conn_->read(_query);
+  }
+
+  Result<Nothing> rollback() noexcept { return conn_->rollback(); }
+
+  std::string to_sql(const dynamic::Statement& _stmt) noexcept {
+    return conn_->to_sql(_stmt);
+  }
+
+  Result<Nothing> start_write(const dynamic::Write& _stmt) {
+    return conn_->start_write(_stmt);
+  }
+
+  Result<Nothing> end_write() { return conn_->end_write(); }
+
+  Result<Nothing> write(
+      const std::vector<std::vector<std::optional<std::string>>>& _data) {
+    return conn_->write(_data);
+  }
+
+ private:
+  /// The underlying connection object.
+  ConnPtr conn_;
+
+  /// The flag corresponding to the object - as long as this is true, we have
+  /// ownership.
+  std::shared_ptr<std::atomic_flag> flag_;
+};
+
+}  // namespace sqlgen
+
+#endif

include/sqlgen/Transaction.hpp

@@ -67,7 +67,7 @@ class Transaction {
     if (this == &_other) {
       return *this;
     }
-    conn_ = _other.conn;
+    conn_ = _other.conn_;
     transaction_ended_ = _other.transaction_ended_;
     _other.transaction_ended_ = true;
     return *this;
@@ -92,7 +92,7 @@ class Transaction {
   }
 
   Result<Nothing> start_write(const dynamic::Write& _stmt) {
-    return conn_->to_sql(_stmt);
+    return conn_->start_write(_stmt);
   }
 
   Result<Nothing> end_write() { return conn_->end_write(); }

tests/postgres/test_write_and_read_pool.cpp

@@ -0,0 +1,57 @@
+#ifndef SQLGEN_BUILD_DRY_TESTS_ONLY
+
+#include <gtest/gtest.h>
+
+#include <rfl.hpp>
+#include <rfl/json.hpp>
+#include <sqlgen.hpp>
+#include <sqlgen/postgres.hpp>
+#include <vector>
+
+namespace test_write_and_read_pool {
+
+struct Person {
+  sqlgen::PrimaryKey<uint32_t> id;
+  std::string first_name;
+  std::string last_name;
+  int age;
+};
+
+TEST(postgres, test_write_and_read_pool) {
+  const auto people1 = std::vector<Person>(
+      {Person{
+           .id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
+       Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10},
+       Person{.id = 2, .first_name = "Lisa", .last_name = "Simpson", .age = 8},
+       Person{
+           .id = 3, .first_name = "Maggie", .last_name = "Simpson", .age = 0}});
+
+  const auto pool_config = sqlgen::ConnectionPoolConfig{.size = 2};
+
+  const auto credentials = sqlgen::postgres::Credentials{.user = "postgres",
+                                                         .password = "password",
+                                                         .host = "localhost",
+                                                         .dbname = "postgres"};
+
+  const auto pool = sqlgen::make_connection_pool<sqlgen::postgres::Connection>(
+      pool_config, credentials);
+
+  using namespace sqlgen;
+
+  const auto people2 = session(pool)
+                           .and_then(drop<Person> | if_exists)
+                           .and_then(write(std::ref(people1)))
+                           .and_then(sqlgen::read<std::vector<Person>>)
+                           .value();
+
+  EXPECT_EQ(pool.value().available(), 2);
+
+  const auto json1 = rfl::json::write(people1);
+  const auto json2 = rfl::json::write(people2);
+
+  EXPECT_EQ(json1, json2);
+}
+
+}  // namespace test_write_and_read_pool
+
+#endif