Minor improvements to dynamic types (#44)

docs/README.md

@@ -41,6 +41,7 @@ Welcome to the sqlgen documentation. This guide provides detailed information ab
 
 ## Data Types and Validation
 
+- [sqlgen::Dynamic](dynamic.md) - How to define custom SQL types not natively supported by sqlgen
 - [sqlgen::ForeignKey](foreign_key.md) - How to establish referential integrity between tables
 - [sqlgen::Pattern](pattern.md) - How to add regex pattern validation to avoid SQL injection
 - [sqlgen::Timestamp](timestamp.md) - How timestamps work in sqlgen

docs/dynamic.md

@@ -0,0 +1,176 @@
+# `sqlgen::Dynamic`
+
+`sqlgen::Dynamic` lets you define custom SQL types that aren't natively supported by sqlgen. It works by returning a `sqlgen::dynamic::types::Dynamic` with a database type name string that the transpiler passes directly to the target database.
+
+## Usage
+
+### Parser specialization for boost::uuids::uuid
+
+In this example, we demonstrate how you can use boost::uuids::uuid to automatically generate primary keys. This is not officially supported by the sqlgen library,
+but it is very easy to build something like this using `sqlgen::Dynamic`.
+
+The first step is to specialize the `sqlgen::parsing::Parser` for `boost::uuids::uuid` and implement `read`, `write`, and `to_type`:
+
+```cpp
+#include <boost/lexical_cast.hpp>
+#include <boost/uuid/uuid.hpp>
+#include <boost/uuid/uuid_io.hpp>
+#include <sqlgen/dynamic/types.hpp>
+#include <sqlgen/parsing/Parser.hpp>
+
+namespace sqlgen::parsing {
+
+template <>
+struct Parser<boost::uuids::uuid> {
+  using Type = boost::uuids::uuid;
+
+  static Result<boost::uuids::uuid> read(
+      const std::optional<std::string>& _str) {
+    if (!_str) {
+      return error("boost::uuids::uuid cannot be NULL.");
+    }
+    return boost::lexical_cast<boost::uuids::uuid>(*_str);
+  }
+
+  static std::optional<std::string> write(
+      const boost::uuids::uuid& _u) {
+    return boost::uuids::to_string(_u);
+  }
+
+  static dynamic::Type to_type() noexcept {
+    return sqlgen::dynamic::types::Dynamic{"UUID"};
+  }
+};
+
+}  // namespace sqlgen::parsing
+```
+
+### Using `boost::uuids::uuid` in structs
+
+You can then automatically generate random UUIDs:
+
+```cpp
+#include <boost/uuid/random_generator.hpp>
+#include <boost/uuid/uuid.hpp>
+
+struct Person {
+  sqlgen::PrimaryKey<boost::uuids::uuid> id =
+      boost::uuids::uuid(boost::uuids::random_generator()());
+  std::string first_name;
+  std::string last_name;
+  int age;
+};
+```
+
+This generates SQL schema with the custom UUID type name:
+
+```sql
+CREATE TABLE IF NOT EXISTS "Person"(
+    "id"         UUID NOT NULL,
+    "first_name" TEXT NOT NULL,
+    "last_name"  TEXT NOT NULL,
+    "age"        INTEGER NOT NULL,
+    PRIMARY KEY("id")
+);
+```
+
+### Working with UUIDs
+
+Note that you do not have to assign the UUIDs - this is done automatically:
+
+```cpp
+// Create and insert records
+std::vector<Person> people = {
+  Person{.first_name = "Homer",  .last_name = "Simpson", .age = 45},
+  Person{.first_name = "Marge",  .last_name = "Simpson", .age = 42},
+  Person{.first_name = "Bart",   .last_name = "Simpson", .age = 10},
+  Person{.first_name = "Lisa",   .last_name = "Simpson", .age = 8},
+  Person{.first_name = "Maggie", .last_name = "Simpson", .age = 0}
+};
+
+// Create table and write
+auto res = conn.and_then(sqlgen::create_table<Person> | sqlgen::if_not_exists)
+               .and_then(sqlgen::insert(std::ref(people)));
+
+// Read back ordered by age
+using namespace sqlgen::literals;
+const auto people2 = res.and_then(sqlgen::read<std::vector<Person>> |
+                                  sqlgen::order_by("age"_c.desc()))
+                        .value();
+
+// Filtering by UUID
+const auto target = boost::lexical_cast<boost::uuids::uuid>(
+    "550e8400-e29b-41d4-a716-446655440000");
+const auto query = sqlgen::read<std::vector<Person>> |
+                   sqlgen::where("id"_c == target);
+```
+
+## Per-database type name for UUID
+
+You may want to map `boost::uuids::uuid` to different database type names per dialect. Implement `to_type()` accordingly. The tests demonstrate these mappings:
+
+- SQLite:
+```cpp
+static dynamic::Type to_type() noexcept {
+  return sqlgen::dynamic::types::Dynamic{"TEXT"};
+}
+```
+
+- PostgreSQL:
+```cpp
+static dynamic::Type to_type() noexcept {
+  return sqlgen::dynamic::types::Dynamic{"UUID"};
+}
+```
+
+- MySQL:
+```cpp
+static dynamic::Type to_type() noexcept {
+  return sqlgen::dynamic::types::Dynamic{"VARCHAR(36)"};
+}
+```
+
+## Parser specialization requirements
+
+Specializing `sqlgen::parsing::Parser<T>` requires three methods. These guidelines help ensure correctness and portability:
+
+1) read
+```cpp
+static Result<T> read(const std::optional<std::string>& dbValue);
+```
+- Responsibility: Convert from DB string (or null) to `T`.
+- Null handling: If your field cannot be null, return `error("... cannot be NULL.")` when `dbValue` is `std::nullopt`.
+- Validation: Parse and validate strictly. Return a descriptive error for malformed input.
+- Normalization: If your string form can vary (case, hyphens), normalize consistently so `write(read(x))` is stable.
+
+2) write
+```cpp
+static std::optional<std::string> write(const T& value);
+```
+- Responsibility: Convert `T` to the string the DB expects.
+- Nulls: Return `std::nullopt` only if you intend to write SQL NULL.
+- Round-trip: Aim for `read(write(v)) == v` (modulo normalization).
+
+3) to_type
+```cpp
+static dynamic::Type to_type() noexcept;
+```
+- Responsibility: Provide the DB type name via `sqlgen::dynamic::types::Dynamic{"TYPE_NAME"}`.
+- Dialect mapping: Choose a valid name for your target DB (e.g., `UUID` on PostgreSQL, `VARCHAR(36)` on MySQL, `TEXT` on SQLite for UUIDs).
+- Column properties: Constraints like primary key, unique, and nullability are typically controlled by field wrappers (`sqlgen::PrimaryKey`, `sqlgen::Unique`, `std::optional<T>`). If you are building fully dynamic schemas, you may also set properties on `Dynamic`.
+
+Additional best practices:
+- Error messages: Keep them clear and specific to aid debugging.
+- Performance: Prefer lightweight conversions in `read`/`write`; avoid expensive allocations inside hot loops.
+- Testing: Add round-trip tests (insert/read/compare) like the provided UUID tests across dialects.
+- Consistency: Ensure the chosen DB type name matches any indexes, constraints, and length limits you rely on.
+
+## Notes
+
+- `sqlgen::dynamic::types::Dynamic` has:
+  - `type_name`: SQL type name string
+  - `properties`: column properties (nullable, unique, primary, auto_incr, foreign_key_reference)
+- Works with all operations: `create_table`, `insert`, `select`, `update`, `delete`
+- The type name is passed directly to the database; ensure it is valid for the target dialect
+- Keep specializations in the `sqlgen::parsing` namespace
+

include/sqlgen/dynamic/Type.hpp

@@ -9,11 +9,11 @@
 namespace sqlgen::dynamic {
 
 using Type =
-    rfl::TaggedUnion<"type", types::Unknown, types::Boolean, types::Float32,
-                     types::Float64, types::Int8, types::Int16, types::Int32,
-                     types::Int64, types::UInt8, types::UInt16, types::UInt32,
-                     types::UInt64, types::Text, types::Date, types::Timestamp,
-                     types::TimestampWithTZ, types::VarChar, types::Dynamic>;
+    rfl::TaggedUnion<"type", types::Unknown, types::Boolean, types::Dynamic,
+                     types::Float32, types::Float64, types::Int8, types::Int16,
+                     types::Int32, types::Int64, types::UInt8, types::UInt16,
+                     types::UInt32, types::UInt64, types::Text, types::Date,
+                     types::Timestamp, types::TimestampWithTZ, types::VarChar>;
 
 }  // namespace sqlgen::dynamic
 

include/sqlgen/dynamic/types.hpp

@@ -29,6 +29,11 @@ struct Boolean {
   Properties properties;
 };
 
+struct Dynamic {
+  std::string type_name;
+  Properties properties;
+};
+
 struct Float32 {
   Properties properties;
 };
@@ -93,11 +98,6 @@ struct VarChar {
   Properties properties;
 };
 
-struct Dynamic {
-  std::string type_name;
-  Properties properties;
-};
-
 }  // namespace sqlgen::dynamic::types
 
 #endif

src/sqlgen/mysql/to_sql.cpp

@@ -120,6 +120,9 @@ std::string cast_type_to_sql(const dynamic::Type& _type) noexcept {
     if constexpr (std::is_same_v<T, dynamic::types::Boolean>) {
       return "BOOLEAN";
 
+    } else if constexpr (std::is_same_v<T, dynamic::types::Dynamic>) {
+      return _t.type_name;
+
     } else if constexpr (std::is_same_v<T, dynamic::types::Int8> ||
                          std::is_same_v<T, dynamic::types::Int16> ||
                          std::is_same_v<T, dynamic::types::Int32> ||
@@ -833,6 +836,9 @@ std::string type_to_sql(const dynamic::Type& _type) noexcept {
     if constexpr (std::is_same_v<T, dynamic::types::Boolean>) {
       return "BOOLEAN";
 
+    } else if constexpr (std::is_same_v<T, dynamic::types::Dynamic>) {
+      return _t.type_name;
+
     } else if constexpr (std::is_same_v<T, dynamic::types::Int8>) {
       return "TINYINT";
 
@@ -865,8 +871,6 @@ std::string type_to_sql(const dynamic::Type& _type) noexcept {
     } else if constexpr (std::is_same_v<T, dynamic::types::Timestamp> ||
                          std::is_same_v<T, dynamic::types::TimestampWithTZ>) {
       return "DATETIME";
-    } else if constexpr (std::is_same_v<T, dynamic::types::Dynamic>) {
-      return _t.type_name;
 
     } else if constexpr (std::is_same_v<T, dynamic::types::Unknown>) {
       return "TEXT";

src/sqlgen/postgres/to_sql.cpp

@@ -718,6 +718,9 @@ std::string type_to_sql(const dynamic::Type& _type) noexcept {
     if constexpr (std::is_same_v<T, dynamic::types::Boolean>) {
       return "BOOLEAN";
 
+    } else if constexpr (std::is_same_v<T, dynamic::types::Dynamic>) {
+      return _t.type_name;
+
     } else if constexpr (std::is_same_v<T, dynamic::types::Int8> ||
                          std::is_same_v<T, dynamic::types::Int16> ||
                          std::is_same_v<T, dynamic::types::UInt8> ||
@@ -751,9 +754,6 @@ std::string type_to_sql(const dynamic::Type& _type) noexcept {
     } else if constexpr (std::is_same_v<T, dynamic::types::TimestampWithTZ>) {
       return "TIMESTAMP WITH TIME ZONE";
 
-    } else if constexpr (std::is_same_v<T, dynamic::types::Dynamic>) {
-      return _t.type_name;
-
     } else if constexpr (std::is_same_v<T, dynamic::types::Unknown>) {
       return "TEXT";
     } else {

src/sqlgen/sqlite/to_sql.cpp

@@ -720,6 +720,7 @@ std::string to_sql_impl(const dynamic::Statement& _stmt) noexcept {
 std::string type_to_sql(const dynamic::Type& _type) noexcept {
   return _type.visit([](const auto _t) -> std::string {
     using T = std::remove_cvref_t<decltype(_t)>;
+
     if constexpr (std::is_same_v<T, dynamic::types::Boolean> ||
                   std::is_same_v<T, dynamic::types::Int8> ||
                   std::is_same_v<T, dynamic::types::Int16> ||
@@ -730,9 +731,11 @@ std::string type_to_sql(const dynamic::Type& _type) noexcept {
                   std::is_same_v<T, dynamic::types::UInt32> ||
                   std::is_same_v<T, dynamic::types::UInt64>) {
       return "INTEGER";
+
     } else if constexpr (std::is_same_v<T, dynamic::types::Float32> ||
                          std::is_same_v<T, dynamic::types::Float64>) {
       return "REAL";
+
     } else if constexpr (std::is_same_v<T, dynamic::types::Unknown> ||
                          std::is_same_v<T, dynamic::types::Text> ||
                          std::is_same_v<T, dynamic::types::VarChar> ||
@@ -740,8 +743,10 @@ std::string type_to_sql(const dynamic::Type& _type) noexcept {
                          std::is_same_v<T, dynamic::types::Timestamp> ||
                          std::is_same_v<T, dynamic::types::TimestampWithTZ>) {
       return "TEXT";
+
     } else if constexpr (std::is_same_v<T, dynamic::types::Dynamic>) {
       return _t.type_name;
+
     } else {
       static_assert(rfl::always_false_v<T>, "Not all cases were covered.");
     }

tests/mysql/test_dynamic_type.cpp

@@ -0,0 +1,101 @@
+#ifndef SQLGEN_BUILD_DRY_TESTS_ONLY
+
+#include <gtest/gtest.h>
+
+#include <boost/lexical_cast.hpp>
+#include <boost/uuid/random_generator.hpp>
+#include <boost/uuid/uuid.hpp>
+#include <boost/uuid/uuid_io.hpp>
+#include <rfl.hpp>
+#include <rfl/json.hpp>
+#include <sqlgen.hpp>
+#include <sqlgen/mysql.hpp>
+#include <vector>
+
+namespace sqlgen::parsing {
+
+template <>
+struct Parser<boost::uuids::uuid> {
+  using Type = boost::uuids::uuid;
+
+  static Result<boost::uuids::uuid> read(
+      const std::optional<std::string>& _str) noexcept {
+    if (!_str) {
+      return error("boost::uuids::uuid cannot be NULL.");
+    }
+    return boost::lexical_cast<boost::uuids::uuid>(*_str);
+  }
+
+  static std::optional<std::string> write(
+      const boost::uuids::uuid& _u) noexcept {
+    return boost::uuids::to_string(_u);
+  }
+
+  static dynamic::Type to_type() noexcept {
+    return sqlgen::dynamic::types::Dynamic{"VARCHAR(36)"};
+  }
+};
+
+}  // namespace sqlgen::parsing
+
+/// For the JSON serialization - not needed for
+/// the actual DB operations.
+namespace rfl {
+
+template <>
+struct Reflector<boost::uuids::uuid> {
+  using ReflType = std::string;
+
+  static boost::uuids::uuid to(const std::string& _str) {
+    return boost::lexical_cast<boost::uuids::uuid>(_str);
+  }
+
+  static std::string from(const boost::uuids::uuid& _u) {
+    return boost::uuids::to_string(_u);
+  }
+};
+
+}  // namespace rfl
+
+namespace test_dynamic_type {
+
+struct Person {
+  sqlgen::PrimaryKey<boost::uuids::uuid> id =
+      boost::uuids::uuid(boost::uuids::random_generator()());
+  std::string first_name;
+  std::string last_name;
+  int age;
+};
+
+TEST(mysql, test_dynamic_type) {
+  const auto people1 = std::vector<Person>(
+      {Person{.first_name = "Homer", .last_name = "Simpson", .age = 45},
+       Person{.first_name = "Bart", .last_name = "Simpson", .age = 10},
+       Person{.first_name = "Lisa", .last_name = "Simpson", .age = 8},
+       Person{.first_name = "Maggie", .last_name = "Simpson", .age = 0}});
+
+  const auto credentials = sqlgen::mysql::Credentials{.host = "localhost",
+                                                      .user = "sqlgen",
+                                                      .password = "password",
+                                                      .dbname = "mysql"};
+
+  using namespace sqlgen;
+  using namespace sqlgen::literals;
+
+  const auto conn =
+      mysql::connect(credentials).and_then(drop<Person> | if_exists);
+
+  const auto people2 = sqlgen::write(conn, people1)
+                           .and_then(sqlgen::read<std::vector<Person>> |
+                                     order_by("age"_c.desc()))
+                           .value();
+
+  const auto json1 = rfl::json::write(people1);
+  const auto json2 = rfl::json::write(people2);
+
+  EXPECT_EQ(json1, json2);
+}
+
+}  // namespace test_dynamic_type
+
+#endif

tests/postgres/test_dynamic_type.cpp

@@ -0,0 +1,101 @@
+#ifndef SQLGEN_BUILD_DRY_TESTS_ONLY
+
+#include <gtest/gtest.h>
+
+#include <boost/lexical_cast.hpp>
+#include <boost/uuid/random_generator.hpp>
+#include <boost/uuid/uuid.hpp>
+#include <boost/uuid/uuid_io.hpp>
+#include <rfl.hpp>
+#include <rfl/json.hpp>
+#include <sqlgen.hpp>
+#include <sqlgen/postgres.hpp>
+#include <vector>
+
+namespace sqlgen::parsing {
+
+template <>
+struct Parser<boost::uuids::uuid> {
+  using Type = boost::uuids::uuid;
+
+  static Result<boost::uuids::uuid> read(
+      const std::optional<std::string>& _str) noexcept {
+    if (!_str) {
+      return error("boost::uuids::uuid cannot be NULL.");
+    }
+    return boost::lexical_cast<boost::uuids::uuid>(*_str);
+  }
+
+  static std::optional<std::string> write(
+      const boost::uuids::uuid& _u) noexcept {
+    return boost::uuids::to_string(_u);
+  }
+
+  static dynamic::Type to_type() noexcept {
+    return sqlgen::dynamic::types::Dynamic{"UUID"};
+  }
+};
+
+}  // namespace sqlgen::parsing
+
+/// For the JSON serialization - not needed for
+/// the actual DB operations.
+namespace rfl {
+
+template <>
+struct Reflector<boost::uuids::uuid> {
+  using ReflType = std::string;
+
+  static boost::uuids::uuid to(const std::string& _str) {
+    return boost::lexical_cast<boost::uuids::uuid>(_str);
+  }
+
+  static std::string from(const boost::uuids::uuid& _u) {
+    return boost::uuids::to_string(_u);
+  }
+};
+
+}  // namespace rfl
+
+namespace test_dynamic_type {
+
+struct Person {
+  sqlgen::PrimaryKey<boost::uuids::uuid> id =
+      boost::uuids::uuid(boost::uuids::random_generator()());
+  std::string first_name;
+  std::string last_name;
+  int age;
+};
+
+TEST(postgres, test_dynamic_type) {
+  const auto people1 = std::vector<Person>(
+      {Person{.first_name = "Homer", .last_name = "Simpson", .age = 45},
+       Person{.first_name = "Bart", .last_name = "Simpson", .age = 10},
+       Person{.first_name = "Lisa", .last_name = "Simpson", .age = 8},
+       Person{.first_name = "Maggie", .last_name = "Simpson", .age = 0}});
+
+  const auto credentials = sqlgen::postgres::Credentials{.user = "postgres",
+                                                         .password = "password",
+                                                         .host = "localhost",
+                                                         .dbname = "postgres"};
+
+  using namespace sqlgen;
+  using namespace sqlgen::literals;
+
+  const auto conn =
+      postgres::connect(credentials).and_then(drop<Person> | if_exists);
+
+  const auto people2 = sqlgen::write(conn, people1)
+                           .and_then(sqlgen::read<std::vector<Person>> |
+                                     order_by("age"_c.desc()))
+                           .value();
+
+  const auto json1 = rfl::json::write(people1);
+  const auto json2 = rfl::json::write(people2);
+
+  EXPECT_EQ(json1, json2);
+}
+
+}  // namespace test_dynamic_type
+
+#endif

tests/sqlite/test_dynamic_type.cpp

@@ -0,0 +1,92 @@
+#include <gtest/gtest.h>
+
+#include <boost/lexical_cast.hpp>
+#include <boost/uuid/random_generator.hpp>
+#include <boost/uuid/uuid.hpp>
+#include <boost/uuid/uuid_io.hpp>
+#include <rfl.hpp>
+#include <rfl/json.hpp>
+#include <sqlgen.hpp>
+#include <sqlgen/sqlite.hpp>
+#include <vector>
+
+namespace sqlgen::parsing {
+
+template <>
+struct Parser<boost::uuids::uuid> {
+  using Type = boost::uuids::uuid;
+
+  static Result<boost::uuids::uuid> read(
+      const std::optional<std::string>& _str) noexcept {
+    if (!_str) {
+      return error("boost::uuids::uuid cannot be NULL.");
+    }
+    return boost::lexical_cast<boost::uuids::uuid>(*_str);
+  }
+
+  static std::optional<std::string> write(
+      const boost::uuids::uuid& _u) noexcept {
+    return boost::uuids::to_string(_u);
+  }
+
+  static dynamic::Type to_type() noexcept {
+    return sqlgen::dynamic::types::Dynamic{"TEXT"};
+  }
+};
+
+}  // namespace sqlgen::parsing
+
+/// For the JSON serialization - not needed for
+/// the actual DB operations.
+namespace rfl {
+
+template <>
+struct Reflector<boost::uuids::uuid> {
+  using ReflType = std::string;
+
+  static boost::uuids::uuid to(const std::string& _str) {
+    return boost::lexical_cast<boost::uuids::uuid>(_str);
+  }
+
+  static std::string from(const boost::uuids::uuid& _u) {
+    return boost::uuids::to_string(_u);
+  }
+};
+
+}  // namespace rfl
+
+namespace test_dynamic_type {
+
+struct Person {
+  sqlgen::PrimaryKey<boost::uuids::uuid> id =
+      boost::uuids::uuid(boost::uuids::random_generator()());
+  std::string first_name;
+  std::string last_name;
+  int age;
+};
+
+TEST(sqlite, test_dynamic_type) {
+  const auto people1 = std::vector<Person>(
+      {Person{.first_name = "Homer", .last_name = "Simpson", .age = 45},
+       Person{.first_name = "Bart", .last_name = "Simpson", .age = 10},
+       Person{.first_name = "Lisa", .last_name = "Simpson", .age = 8},
+       Person{.first_name = "Maggie", .last_name = "Simpson", .age = 0}});
+
+  using namespace sqlgen;
+  using namespace sqlgen::literals;
+
+  const auto conn = sqlite::connect().and_then(drop<Person> | if_exists);
+
+  const auto people2 = sqlgen::write(conn, people1)
+                           .and_then(sqlgen::read<std::vector<Person>> |
+                                     order_by("age"_c.desc()))
+                           .value();
+
+  const auto json1 = rfl::json::write(people1);
+  const auto json2 = rfl::json::write(people2);
+
+  EXPECT_EQ(json1, json2);
+}
+
+}  // namespace test_dynamic_type
+

vcpkg.json

@@ -40,6 +40,14 @@
         {
           "name": "gtest",
           "version>=": "1.14.0"
+        },
+        {
+          "name": "boost-lexical-cast",
+          "version>=": "1.88.0"
+        },
+        {
+          "name": "boost-uuid",
+          "version>=": "1.88.0"
         }
       ]
     }