mirror/sqlgen
1
0
Fork 0
mirror of https://github.com/getml/sqlgen.git synced 2026-02-14 12:48:39 -06:00
Code Issues Packages Projects Releases Wiki Activity

Added a separate insert statement (#3)

Browse Source
This commit is contained in:
Dr. Patrick Urbanke (劉自成)
2025-05-21 22:28:02 +02:00
committed by GitHub
parent 7396ae39b0
commit 3318c40c04
27 changed files with 893 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
README.md
View File

@@ -4,6 +4,7 @@
[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://github.com/getml/reflect-cpp/graphs/commit-activity)
[![Generic badge](https://img.shields.io/badge/C++-20-blue.svg)](https://shields.io/)
[![Generic badge](https://img.shields.io/badge/gcc-11+-blue.svg)](https://shields.io/)
[![Generic badge](https://img.shields.io/badge/clang-14+-blue.svg)](https://shields.io/)
sqlgen is a modern, type-safe ORM and SQL query generator for C++20, inspired by Python's [SQLAlchemy](https://github.com/sqlalchemy/sqlalchemy)/[SQLModel](https://github.com/fastapi/sqlmodel) and Rust's [Diesel](https://github.com/diesel-rs/diesel). It provides a fluent, composable interface for database operations with compile-time type checking and SQL injection protection.

7
docs/README.md
View File

@@ -14,11 +14,14 @@ Welcome to the sqlgen documentation. This guide provides detailed information ab
- [sqlgen::read](reading.md) - How to read data from a database
- [sqlgen::write](writing.md) - How to write data to a database
- [sqlgen::create_index](create_index.md) - How to create an index on a table
- [sqlgen::create_table](create_table.md) - How to create a new table
- [sqlgen::update](update.md) - How to update data in a table
- [sqlgen::delete_from](delete_from.md) - How to delete data from a table
- [sqlgen::drop](drop.md) - How to drop a table
- [sqlgen::create_index](create_index.md) - How to create an index on a table
- [sqlgen::insert](insert.md) - How to insert data within transactions
- [sqlgen::update](update.md) - How to update data in a table
- [Transactions](transactions.md) - How to use transactions for atomic operations
## Data Types and Validation

199
docs/insert.md Normal file
View File

@@ -0,0 +1,199 @@
# `sqlgen::insert`
The `sqlgen::insert` interface provides a type-safe way to insert data from C++ containers or ranges into a SQL database. Unlike `sqlgen::write`, it does not create tables automatically and is designed to be used within transactions. It's particularly useful when you need fine-grained control over table creation and transaction boundaries.
In particular, `sqlgen::insert` is the recommended way whenever you want to insert data into several tables that depend on each other, meaning that either all inserts should succeed or none of them.
## Usage
### Basic Insert
Insert a container of objects into a database:
```cpp
const auto people = std::vector<Person>({
Person{.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10}
});
// Using with a connection reference
const auto conn = sqlgen::sqlite::connect();
sqlgen::insert(conn, people);
```
This generates the following SQL:
```sql
INSERT INTO "Person" ("id", "first_name", "last_name", "age") VALUES (?, ?, ?, ?);
```
Note that `conn` is actually a connection wrapped into an `sqlgen::Result<...>`.
This means you can use monadic error handling and fit this into a single line:
```cpp
// sqlgen::Result<Ref<Connection>>
const auto result = sqlgen::sqlite::connect("database.db").and_then(
sqlgen::insert(people));
```
Please refer to the documentation on `sqlgen::Result<...>` for more information on error handling.
### With Result<Ref<Connection>>
Handle connection creation and insertion in a single chain:
```cpp
sqlgen::sqlite::connect("database.db")
.and_then(sqlgen::insert(people))
.value();
```
### With Iterators
Insert a range of objects using iterators:
```cpp
std::vector<Person> people = /* ... */;
sqlgen::insert(conn, people.begin(), people.end());
```
### With Reference Wrapper
Insert data using a reference wrapper to avoid copying:
```cpp
const auto people = std::vector<Person>(/* ... */);
sqlgen::sqlite::connect("database.db")
.and_then(sqlgen::insert(std::ref(people)))
.value();```
## Example: Full Transaction Usage
Here's a complete example showing how to use `insert` within a transaction:
```cpp
using namespace sqlgen;
const auto result = sqlite::connect()
.and_then(begin_transaction)
.and_then(create_table<Person> | if_not_exists)
.and_then(insert(std::ref(people)))
.and_then(commit)
.value();
```
This generates the following SQL:
```sql
BEGIN TRANSACTION;
CREATE TABLE IF NOT EXISTS "Person" (
"id" INTEGER PRIMARY KEY,
"first_name" TEXT NOT NULL,
"last_name" TEXT NOT NULL,
"age" INTEGER NOT NULL
);
INSERT INTO "Person" ("id", "first_name", "last_name", "age") VALUES (?, ?, ?, ?);
COMMIT;
```
## Example: Multi-Table Insert with Dependencies
Here's an example showing how to insert related data into multiple tables within a single transaction:
```cpp
struct Person {
sqlgen::PrimaryKey<uint32_t> id;
std::string first_name;
std::string last_name;
int age;
};
struct Children {
uint32_t id_parent;
sqlgen::PrimaryKey<uint32_t> id_child;
};
// Parent records
const auto people = std::vector<Person>({
Person{.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10},
Person{.id = 2, .first_name = "Lisa", .last_name = "Simpson", .age = 8},
Person{.id = 3, .first_name = "Maggie", .last_name = "Simpson", .age = 0}
});
// Child relationships
const auto children = std::vector<Children>({
Children{.id_parent = 0, .id_child = 1}, // Homer -> Bart
Children{.id_parent = 0, .id_child = 2}, // Homer -> Lisa
Children{.id_parent = 0, .id_child = 3} // Homer -> Maggie
});
using namespace sqlgen;
// All inserts are wrapped in a single transaction
const auto result = postgres::connect(credentials)
.and_then(begin_transaction)
.and_then(create_table<Person> | if_not_exists)
.and_then(create_table<Children> | if_not_exists)
.and_then(insert(std::ref(people))) // Insert people first
.and_then(insert(std::ref(children))) // Then insert relationships
.and_then(commit)
.value();
```
This generates the following SQL:
```sql
BEGIN TRANSACTION;
CREATE TABLE IF NOT EXISTS "Person" (
"id" INTEGER PRIMARY KEY,
"first_name" TEXT NOT NULL,
"last_name" TEXT NOT NULL,
"age" INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS "Children" (
"id_parent" INTEGER NOT NULL,
"id_child" INTEGER PRIMARY KEY
);
INSERT INTO "Person" ("id", "first_name", "last_name", "age") VALUES (?, ?, ?, ?);
INSERT INTO "Children" ("id_parent", "id_child") VALUES (?, ?);
COMMIT;
```
In this example, we insert both parent records and their relationships to children. If any insert fails, the entire transaction is rolled back, ensuring data consistency. This is crucial when dealing with related data across multiple tables.
## Comparison with `sqlgen::write`
While both `insert` and `write` can be used to add data to a database, they serve different purposes:
### `sqlgen::write`
- Automatically creates tables if they don't exist
- Uses optimized bulk insert methods (e.g., PostgreSQL's COPY command)
- Handles batching automatically
- More convenient for simple use cases
- Faster for bulk inserts
- Cannot be used within transactions
- Best for single-table operations with no dependencies
### `sqlgen::insert`
- Does not create tables automatically
- Uses standard INSERT statements
- Designed to be used within transactions
- More control over transaction boundaries
- Better for complex operations requiring transaction support
- Essential for multi-table operations with dependencies
- Guarantees atomicity across multiple tables
- Recommended when data consistency across tables is critical
## Notes
- The `Result<Ref<Connection>>` type provides error handling; use `.value()` to extract the result (will throw an exception if there's an error) or handle errors as needed
- The function has several overloads:
1. Takes a connection reference and iterators
2. Takes a `Result<Ref<Connection>>` and iterators
3. Takes a connection and a container directly
4. Takes a connection and a reference wrapper to a container
- Unlike `write`, `insert` does not create tables automatically - you must create tables separately using `create_table`
- The insert operation is atomic within a transaction
- When using reference wrappers (`std::ref`), the data is not copied, which can be more efficient for large datasets

30
docs/writing.md
View File

@@ -68,6 +68,36 @@ sqlgen::write(conn, people.begin(), people.end());
This also generates the same SQL, adapted to the specific database dialect.
### Curried Write
You can also use a curried version of `write` that takes the data first and returns a function that accepts the connection. This is useful for chaining operations:
```cpp
const auto people = std::vector<Person>({
Person{.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10}
});
// Using with a connection reference
const auto conn = sqlgen::sqlite::connect();
sqlgen::write(people)(conn); // Creates a deep copy of people
// To avoid deep copy, use std::ref
sqlgen::write(std::ref(people))(conn); // Passes people by reference
// Or in a chain with other operations
sqlgen::sqlite::connect()
.and_then(sqlgen::write(std::ref(people))) // Pass data by reference
.and_then(sqlgen::read<std::vector<Person>>)
.value();
```
Note that by default, the curried `write` will create a deep copy of your data. If you want to avoid this overhead, wrap your data in `std::ref` when passing it to `write`. This is especially important for large datasets.
The curried version is particularly useful when you want to:
- Chain multiple database operations together
- Pass the write operation as a function to other operations
## How It Works
The `write` function performs the following operations in sequence:

2
include/sqlgen.hpp
View File

@@ -3,7 +3,6 @@
#include "sqlgen/Connection.hpp"
#include "sqlgen/Flatten.hpp"
#include "sqlgen/Insert.hpp"
#include "sqlgen/Iterator.hpp"
#include "sqlgen/IteratorBase.hpp"
#include "sqlgen/Literal.hpp"
@@ -23,6 +22,7 @@
#include "sqlgen/drop.hpp"
#include "sqlgen/if_exists.hpp"
#include "sqlgen/if_not_exists.hpp"
#include "sqlgen/insert.hpp"
#include "sqlgen/limit.hpp"
#include "sqlgen/order_by.hpp"
#include "sqlgen/patterns.hpp"

10
include/sqlgen/Connection.hpp
View File

@@ -8,9 +8,9 @@
#include "IteratorBase.hpp"
#include "Ref.hpp"
#include "Result.hpp"
#include "dynamic/Insert.hpp"
#include "dynamic/SelectFrom.hpp"
#include "dynamic/Statement.hpp"
#include "dynamic/Write.hpp"
namespace sqlgen {
@@ -29,6 +29,12 @@ struct Connection {
/// you must call .commit() afterwards.
virtual Result<Nothing> execute(const std::string& _sql) = 0;
/// Inserts data into the database using the INSERT statement.
/// More minimal approach than write, but can be used inside transactions.
virtual Result<Nothing> insert(
const dynamic::Insert& _stmt,
const std::vector<std::vector<std::optional<std::string>>>& _data) = 0;
/// Reads the results of a SelectFrom statement.
virtual Result<Ref<IteratorBase>> read(const dynamic::SelectFrom& _query) = 0;
@@ -39,7 +45,7 @@ struct Connection {
virtual std::string to_sql(const dynamic::Statement& _stmt) = 0;
/// Starts the write operation.
virtual Result<Nothing> start_write(const dynamic::Insert& _stmt) = 0;
virtual Result<Nothing> start_write(const dynamic::Write& _stmt) = 0;
/// Ends the write operation and thus commits the results.
virtual Result<Nothing> end_write() = 0;

15
include/sqlgen/Insert.hpp
View File

@@ -1,15 +0,0 @@
#ifndef SQLGEN_INSERT_HPP_
#define SQLGEN_INSERT_HPP_
#include <rfl.hpp>
namespace sqlgen {
/// Helper class for to_sql.
template <class T>
struct Insert {};
}; // namespace sqlgen
#endif

3
include/sqlgen/dynamic/Statement.hpp
View File

@@ -10,11 +10,12 @@
#include "Insert.hpp"
#include "SelectFrom.hpp"
#include "Update.hpp"
#include "Write.hpp"
namespace sqlgen::dynamic {
using Statement = rfl::TaggedUnion<"stmt", CreateIndex, CreateTable, DeleteFrom,
Drop, Insert, SelectFrom, Update>;
Drop, Insert, SelectFrom, Update, Write>;
} // namespace sqlgen::dynamic

18
include/sqlgen/dynamic/Write.hpp Normal file
View File

@@ -0,0 +1,18 @@
#ifndef SQLGEN_DYNAMIC_WRITE_HPP_
#define SQLGEN_DYNAMIC_WRITE_HPP_
#include <string>
#include <vector>
#include "Table.hpp"
namespace sqlgen::dynamic {
struct Write {
Table table;
std::vector<std::string> columns;
};
} // namespace sqlgen::dynamic
#endif

80
include/sqlgen/insert.hpp Normal file
View File

@@ -0,0 +1,80 @@
#ifndef SQLGEN_INSERT_HPP_
#define SQLGEN_INSERT_HPP_
#include <functional>
#include <iterator>
#include <optional>
#include <rfl.hpp>
#include <string>
#include <type_traits>
#include <vector>
#include "internal/batch_size.hpp"
#include "internal/to_str_vec.hpp"
#include "transpilation/to_insert_or_write.hpp"
namespace sqlgen {
template <class ItBegin, class ItEnd>
Result<Ref<Connection>> insert(const Ref<Connection>& _conn, ItBegin _begin,
ItEnd _end) {
using T =
std::remove_cvref_t<typename std::iterator_traits<ItBegin>::value_type>;
const auto insert_stmt =
transpilation::to_insert_or_write<T, dynamic::Insert>();
std::vector<std::vector<std::optional<std::string>>> data;
for (auto it = _begin; it != _end; ++it) {
data.emplace_back(internal::to_str_vec(*it));
}
return _conn->insert(insert_stmt, data).transform([&](const auto&) {
return _conn;
});
}
template <class ItBegin, class ItEnd>
Result<Ref<Connection>> insert(const Result<Ref<Connection>>& _res,
ItBegin _begin, ItEnd _end) {
return _res.and_then(
[&](const auto& _conn) { return insert(_conn, _begin, _end); });
}
template <class ContainerType>
Result<Ref<Connection>> insert(const auto& _conn, const ContainerType& _data) {
if constexpr (std::ranges::input_range<std::remove_cvref_t<ContainerType>>) {
return insert(_conn, _data.begin(), _data.end());
} else {
return insert(_conn, &_data, &_data + 1);
}
}
template <class ContainerType>
Result<Ref<Connection>> insert(
const auto& _conn, const std::reference_wrapper<ContainerType>& _data) {
return insert(_conn, _data.get());
}
template <class ContainerType>
struct Insert {
Result<Ref<Connection>> operator()(const auto& _conn) const noexcept {
try {
return insert(_conn, data_);
} catch (std::exception& e) {
return error(e.what());
}
}
ContainerType data_;
};
template <class ContainerType>
Insert<ContainerType> insert(const ContainerType& _data) {
return Insert<ContainerType>{.data_ = _data};
}
}; // namespace sqlgen
#endif

8
include/sqlgen/postgres/Connection.hpp
View File

@@ -14,6 +14,7 @@
#include "../Result.hpp"
#include "../dynamic/Column.hpp"
#include "../dynamic/Statement.hpp"
#include "../dynamic/Write.hpp"
#include "Credentials.hpp"
#include "exec.hpp"
#include "to_sql.hpp"
@@ -46,6 +47,11 @@ class Connection : public sqlgen::Connection {
return exec(conn_, _sql).transform([](auto&&) { return Nothing{}; });
}
Result<Nothing> insert(
const dynamic::Insert& _stmt,
const std::vector<std::vector<std::optional<std::string>>>&
_data) noexcept final;
Connection& operator=(const Connection& _other) = delete;
Connection& operator=(Connection&& _other) noexcept;
@@ -58,7 +64,7 @@ class Connection : public sqlgen::Connection {
return postgres::to_sql_impl(_stmt);
}
Result<Nothing> start_write(const dynamic::Insert& _stmt) final {
Result<Nothing> start_write(const dynamic::Write& _stmt) final {
return execute(postgres::to_sql_impl(_stmt));
}

17
include/sqlgen/sqlite/Connection.hpp
View File

@@ -13,6 +13,7 @@
#include "../IteratorBase.hpp"
#include "../Ref.hpp"
#include "../Result.hpp"
#include "../dynamic/Write.hpp"
#include "to_sql.hpp"
namespace sqlgen::sqlite {
@@ -40,6 +41,11 @@ class Connection : public sqlgen::Connection {
Result<Nothing> execute(const std::string& _sql) noexcept final;
Result<Nothing> insert(
const dynamic::Insert& _stmt,
const std::vector<std::vector<std::optional<std::string>>>&
_data) noexcept final;
Connection& operator=(const Connection& _other) = delete;
Connection& operator=(Connection&& _other) noexcept;
@@ -52,7 +58,7 @@ class Connection : public sqlgen::Connection {
return sqlite::to_sql_impl(_stmt);
}
Result<Nothing> start_write(const dynamic::Insert& _stmt) final;
Result<Nothing> start_write(const dynamic::Write& _stmt) final;
Result<Nothing> end_write() final;
@@ -63,6 +69,15 @@ class Connection : public sqlgen::Connection {
/// Generates the underlying connection.
static ConnPtr make_conn(const std::string& _fname);
/// Actually inserts data based on a prepared statement -
/// used by both .insert(...) and .write(...).
Result<Nothing> actual_insert(
const std::vector<std::vector<std::optional<std::string>>>& _data,
sqlite3_stmt* _stmt) const noexcept;
/// Generates a prepared statment, usually for inserts.
Result<StmtPtr> prepare_statement(const std::string& _sql) const noexcept;
private:
/// A prepared statement - needed for the read and write operations. Note that
/// we have declared it before conn_, meaning it will be destroyed first.

17
include/sqlgen/transpilation/to_insert.hpp → include/sqlgen/transpilation/to_insert_or_write.hpp
View File

@@ -1,5 +1,5 @@
#ifndef SQLGEN_TRANSPILATION_TO_INSERT_HPP_
#define SQLGEN_TRANSPILATION_TO_INSERT_HPP_
#ifndef SQLGEN_TRANSPILATION_TO_INSERT_OR_WRITE_HPP_
#define SQLGEN_TRANSPILATION_TO_INSERT_OR_WRITE_HPP_
#include <ranges>
#include <rfl.hpp>
@@ -8,7 +8,6 @@
#include <utility>
#include <vector>
#include "../dynamic/Insert.hpp"
#include "../dynamic/Table.hpp"
#include "../internal/collect/vector.hpp"
#include "get_schema.hpp"
@@ -17,10 +16,10 @@
namespace sqlgen::transpilation {
template <class T>
template <class T, class InsertOrWrite>
requires std::is_class_v<std::remove_cvref_t<T>> &&
std::is_aggregate_v<std::remove_cvref_t<T>>
dynamic::Insert to_insert() {
InsertOrWrite to_insert_or_write() {
using namespace std::ranges::views;
using NamedTupleType = rfl::named_tuple_t<std::remove_cvref_t<T>>;
@@ -31,10 +30,10 @@ dynamic::Insert to_insert() {
const auto get_name = [](const auto& _col) { return _col.name; };
return dynamic::Insert{.table = dynamic::Table{.name = get_tablename<T>(),
.schema = get_schema<T>()},
.columns = sqlgen::internal::collect::vector(
columns | transform(get_name))};
return InsertOrWrite{.table = dynamic::Table{.name = get_tablename<T>(),
.schema = get_schema<T>()},
.columns = sqlgen::internal::collect::vector(
columns | transform(get_name))};
}
} // namespace sqlgen::transpilation

8
include/sqlgen/transpilation/to_sql.hpp
View File

@@ -3,12 +3,12 @@
#include <vector>
#include "../Insert.hpp"
#include "../create_index.hpp"
#include "../create_table.hpp"
#include "../delete_from.hpp"
#include "../drop.hpp"
#include "../dynamic/Statement.hpp"
#include "../insert.hpp"
#include "../read.hpp"
#include "../update.hpp"
#include "columns_t.hpp"
@@ -16,7 +16,7 @@
#include "to_create_table.hpp"
#include "to_delete_from.hpp"
#include "to_drop.hpp"
#include "to_insert.hpp"
#include "to_insert_or_write.hpp"
#include "to_select_from.hpp"
#include "to_update.hpp"
#include "value_t.hpp"
@@ -60,7 +60,9 @@ struct ToSQL<Drop<T>> {
template <class T>
struct ToSQL<Insert<T>> {
dynamic::Statement operator()(const auto&) const { return to_insert<T>(); }
dynamic::Statement operator()(const auto&) const {
return to_insert_or_write<T, dynamic::Insert>();
}
};
template <class ContainerType, class WhereType, class OrderByType,

42
include/sqlgen/write.hpp
View File

@@ -1,6 +1,7 @@
#ifndef SQLGEN_WRITE_HPP_
#define SQLGEN_WRITE_HPP_
#include <functional>
#include <iterator>
#include <optional>
#include <ranges>
@@ -12,10 +13,11 @@
#include "Connection.hpp"
#include "Ref.hpp"
#include "Result.hpp"
#include "dynamic/Write.hpp"
#include "internal/batch_size.hpp"
#include "internal/to_str_vec.hpp"
#include "transpilation/to_create_table.hpp"
#include "transpilation/to_insert.hpp"
#include "transpilation/to_insert_or_write.hpp"
namespace sqlgen {
@@ -26,8 +28,9 @@ Result<Ref<Connection>> write(const Ref<Connection>& _conn, ItBegin _begin,
std::remove_cvref_t<typename std::iterator_traits<ItBegin>::value_type>;
const auto start_write = [&](const auto&) -> Result<Nothing> {
const auto insert_stmt = transpilation::to_insert<T>();
return _conn->start_write(insert_stmt);
const auto write_stmt =
transpilation::to_insert_or_write<T, dynamic::Write>();
return _conn->start_write(write_stmt);
};
const auto write = [&](const auto&) -> Result<Nothing> {
@@ -64,15 +67,15 @@ Result<Ref<Connection>> write(const Ref<Connection>& _conn, ItBegin _begin,
}
template <class ItBegin, class ItEnd>
auto write(const Result<Ref<Connection>>& _res, ItBegin _begin,
ItEnd _end) noexcept {
Result<Ref<Connection>> write(const Result<Ref<Connection>>& _res,
ItBegin _begin, ItEnd _end) noexcept {
return _res.and_then(
[&](const auto& _conn) { return write(_conn, _begin, _end); });
}
template <class ConnectionType, class ContainerType>
auto write(const ConnectionType& _conn,
const ContainerType& _container) noexcept {
Result<Ref<Connection>> write(const ConnectionType& _conn,
const ContainerType& _container) noexcept {
if constexpr (std::ranges::input_range<std::remove_cvref_t<ContainerType>>) {
return write(_conn, _container.begin(), _container.end());
} else {
@@ -80,6 +83,31 @@ auto write(const ConnectionType& _conn,
}
}
template <class ConnectionType, class ContainerType>
Result<Ref<Connection>> write(
const ConnectionType& _conn,
const std::reference_wrapper<ContainerType>& _data) {
return write(_conn, _data.get());
}
template <class ContainerType>
struct Write {
Result<Ref<Connection>> operator()(const auto& _conn) const noexcept {
try {
return write(_conn, data_);
} catch (std::exception& e) {
return error(e.what());
}
}
ContainerType data_;
};
template <class ContainerType>
Write<ContainerType> write(const ContainerType& _data) {
return Write<ContainerType>{.data_ = _data};
}
} // namespace sqlgen
#endif

55
src/sqlgen/postgres/Connection.cpp
View File

@@ -52,6 +52,61 @@ Result<Nothing> Connection::end_write() {
return Nothing{};
}
Result<Nothing> Connection::insert(
const dynamic::Insert& _stmt,
const std::vector<std::vector<std::optional<std::string>>>&
_data) noexcept {
if (_data.size() == 0) {
return Nothing{};
}
const auto sql = to_sql_impl(_stmt);
const auto res = execute("PREPARE \"sqlgen_insert_into_table\" AS " + sql);
if (!res) {
return res;
}
std::vector<const char*> current_row(_data[0].size());
const int n_params = static_cast<int>(current_row.size());
for (size_t i = 0; i < _data.size(); ++i) {
const auto& d = _data[i];
if (d.size() != current_row.size()) {
return error("Error in entry " + std::to_string(i) + ": Expected " +
std::to_string(current_row.size()) + " entries, got " +
std::to_string(d.size()));
}
for (size_t j = 0; j < d.size(); ++j) {
current_row[j] = d[j] ? d[j]->c_str() : nullptr;
}
const auto res = PQexecPrepared(conn_.get(), // conn
"sqlgen_insert_into_table", // stmtName
n_params, // nParams
current_row.data(), // paramValues
nullptr, // paramLengths
nullptr, // paramFormats
0 // resultFormat
);
const auto status = PQresultStatus(res);
if (status != PGRES_COMMAND_OK) {
const auto err = error(std::string("Executing INSERT failed: ") +
PQresultErrorMessage(res));
execute("DEALLOCATE sqlgen_insert_into_table;");
return err;
}
}
return execute("DEALLOCATE sqlgen_insert_into_table;");
}
rfl::Result<Ref<sqlgen::Connection>> Connection::make(
const Credentials& _credentials) noexcept {
try {

45
src/sqlgen/postgres/to_sql.cpp
View File

@@ -41,6 +41,8 @@ std::string type_to_sql(const dynamic::Type& _type) noexcept;
std::string update_to_sql(const dynamic::Update& _stmt) noexcept;
std::string write_to_sql(const dynamic::Write& _stmt) noexcept;
// ----------------------------------------------------------------------------
inline std::string get_name(const dynamic::Column& _col) { return _col.name; }
@@ -260,13 +262,32 @@ std::vector<std::string> get_primary_keys(
std::string insert_to_sql(const dynamic::Insert& _stmt) noexcept {
using namespace std::ranges::views;
const auto schema = wrap_in_quotes(_stmt.table.schema.value_or("public"));
const auto table = wrap_in_quotes(_stmt.table.name);
const auto colnames = internal::strings::join(
const auto to_placeholder = [](const size_t _i) -> std::string {
return "$" + std::to_string(_i + 1);
};
std::stringstream stream;
stream << "INSERT INTO ";
if (_stmt.table.schema) {
stream << "\"" << *_stmt.table.schema << "\".";
}
stream << "\"" << _stmt.table.name << "\"";
stream << " (";
stream << internal::strings::join(
", ",
internal::collect::vector(_stmt.columns | transform(wrap_in_quotes)));
return "COPY " + schema + "." + table + "(" + colnames +
") FROM STDIN WITH DELIMITER '\t' NULL '\e' CSV QUOTE '\a';";
stream << ")";
stream << " VALUES (";
stream << internal::strings::join(
", ", internal::collect::vector(
iota(static_cast<size_t>(0), _stmt.columns.size()) |
transform(to_placeholder)));
stream << ");";
return stream.str();
}
std::string select_from_to_sql(const dynamic::SelectFrom& _stmt) noexcept {
@@ -333,6 +354,9 @@ std::string to_sql_impl(const dynamic::Statement& _stmt) noexcept {
} else if constexpr (std::is_same_v<S, dynamic::Update>) {
return update_to_sql(_s);
} else if constexpr (std::is_same_v<S, dynamic::Write>) {
return write_to_sql(_s);
} else {
static_assert(rfl::always_false_v<S>, "Unsupported type.");
}
@@ -406,4 +430,15 @@ std::string update_to_sql(const dynamic::Update& _stmt) noexcept {
return stream.str();
}
std::string write_to_sql(const dynamic::Write& _stmt) noexcept {
using namespace std::ranges::views;
const auto schema = wrap_in_quotes(_stmt.table.schema.value_or("public"));
const auto table = wrap_in_quotes(_stmt.table.name);
const auto colnames = internal::strings::join(
", ",
internal::collect::vector(_stmt.columns | transform(wrap_in_quotes)));
return "COPY " + schema + "." + table + "(" + colnames +
") FROM STDIN WITH DELIMITER '\t' NULL '\e' CSV QUOTE '\a';";
}
} // namespace sqlgen::postgres

142
src/sqlgen/sqlite/Connection.cpp
View File

@@ -24,6 +24,47 @@ Connection::~Connection() {
}
}
Result<Nothing> Connection::actual_insert(
const std::vector<std::vector<std::optional<std::string>>>& _data,
sqlite3_stmt* _stmt) const noexcept {
for (const auto& row : _data) {
const auto num_cols = static_cast<int>(row.size());
for (int i = 0; i < num_cols; ++i) {
if (row[i]) {
const auto res =
sqlite3_bind_text(_stmt, i + 1, row[i]->c_str(),
static_cast<int>(row[i]->size()), SQLITE_STATIC);
if (res != SQLITE_OK) {
return error(sqlite3_errmsg(conn_.get()));
}
} else {
const auto res = sqlite3_bind_null(_stmt, i + 1);
if (res != SQLITE_OK) {
return error(sqlite3_errmsg(conn_.get()));
}
}
}
auto res = sqlite3_step(_stmt);
if (res != SQLITE_OK && res != SQLITE_ROW && res != SQLITE_DONE) {
return error(sqlite3_errmsg(conn_.get()));
}
res = sqlite3_reset(_stmt);
if (res != SQLITE_OK) {
return error(sqlite3_errmsg(conn_.get()));
}
}
const auto res = sqlite3_clear_bindings(_stmt);
if (res != SQLITE_OK) {
return error(sqlite3_errmsg(conn_.get()));
}
return Nothing{};
}
Result<Nothing> Connection::begin_transaction() noexcept {
if (transaction_started_) {
return error(
@@ -61,6 +102,15 @@ Result<Nothing> Connection::execute(const std::string& _sql) noexcept {
return Nothing{};
}
Result<Nothing> Connection::insert(
const dynamic::Insert& _stmt,
const std::vector<std::vector<std::optional<std::string>>>&
_data) noexcept {
const auto sql = to_sql_impl(_stmt);
return prepare_statement(sql).and_then(
[&](auto _p_stmt) { return actual_insert(_data, _p_stmt.get()); });
}
typename Connection::ConnPtr Connection::make_conn(const std::string& _fname) {
sqlite3* conn = nullptr;
const auto err = sqlite3_open(_fname.c_str(), &conn);
@@ -107,6 +157,25 @@ Result<Ref<IteratorBase>> Connection::read(const dynamic::SelectFrom& _query) {
});
}
Result<Connection::StmtPtr> Connection::prepare_statement(
const std::string& _sql) const noexcept {
sqlite3_stmt* p_stmt = nullptr;
sqlite3_prepare(conn_.get(), /* Database handle */
_sql.c_str(), /* SQL statement, UTF-8 encoded */
_sql.size(), /* Maximum length of zSql in bytes. */
&p_stmt, /* OUT: Statement handle */
nullptr /* OUT: Pointer to unused portion of zSql */
);
if (!p_stmt) {
return error("Preparing the following statement failed: " + _sql +
" Reason: " + sqlite3_errmsg(conn_.get()));
}
return StmtPtr(p_stmt, &sqlite3_finalize);
}
Result<Nothing> Connection::rollback() noexcept {
if (!transaction_started_) {
return error("Cannot ROLLBACK - no transaction has been started.");
@@ -115,7 +184,7 @@ Result<Nothing> Connection::rollback() noexcept {
return execute("ROLLBACK;");
}
Result<Nothing> Connection::start_write(const dynamic::Insert& _stmt) {
Result<Nothing> Connection::start_write(const dynamic::Write& _stmt) {
if (stmt_) {
return error(
"A write operation has already been launched. You need to call "
@@ -124,22 +193,12 @@ Result<Nothing> Connection::start_write(const dynamic::Insert& _stmt) {
const auto sql = to_sql_impl(_stmt);
sqlite3_stmt* p_stmt = nullptr;
sqlite3_prepare(conn_.get(), /* Database handle */
sql.c_str(), /* SQL statement, UTF-8 encoded */
sql.size(), /* Maximum length of zSql in bytes. */
&p_stmt, /* OUT: Statement handle */
nullptr /* OUT: Pointer to unused portion of zSql */
);
if (!p_stmt) {
return error(sqlite3_errmsg(conn_.get()));
}
stmt_ = StmtPtr(p_stmt, &sqlite3_finalize);
return Nothing{};
return prepare_statement(sql)
.transform([&](auto&& _stmt) {
stmt_ = std::move(_stmt);
return Nothing{};
})
.and_then([&](const auto&) { return begin_transaction(); });
}
Result<Nothing> Connection::write(
@@ -150,49 +209,7 @@ Result<Nothing> Connection::write(
".write(...).");
}
const auto write = [&](const auto&) -> Result<Nothing> {
for (const auto& row : _data) {
const auto num_cols = static_cast<int>(row.size());
for (int i = 0; i < num_cols; ++i) {
if (row[i]) {
const auto res = sqlite3_bind_text(
stmt_.get(), i + 1, row[i]->c_str(),
static_cast<int>(row[i]->size()), SQLITE_STATIC);
if (res != SQLITE_OK) {
return error(sqlite3_errmsg(conn_.get()));
}
} else {
const auto res = sqlite3_bind_null(stmt_.get(), i + 1);
if (res != SQLITE_OK) {
return error(sqlite3_errmsg(conn_.get()));
}
}
}
auto res = sqlite3_step(stmt_.get());
if (res != SQLITE_OK && res != SQLITE_ROW && res != SQLITE_DONE) {
return error(sqlite3_errmsg(conn_.get()));
}
res = sqlite3_reset(stmt_.get());
if (res != SQLITE_OK) {
return error(sqlite3_errmsg(conn_.get()));
}
}
// We need to reset the statement to avoid segfaults.
const auto res = sqlite3_clear_bindings(stmt_.get());
if (res != SQLITE_OK) {
return error(sqlite3_errmsg(conn_.get()));
}
return Nothing{};
};
return begin_transaction()
.and_then(write)
.and_then([&](const auto&) { return commit(); })
return actual_insert(_data, stmt_.get())
.or_else([&](const auto& err) -> Result<Nothing> {
rollback();
return error(err.what());
@@ -206,7 +223,10 @@ Result<Nothing> Connection::end_write() {
".end_write().");
}
stmt_ = nullptr;
return Nothing{};
return commit().or_else([&](const auto& err) -> Result<Nothing> {
rollback();
return error(err.what());
});
}
} // namespace sqlgen::sqlite

11
src/sqlgen/sqlite/to_sql.cpp
View File

@@ -26,7 +26,8 @@ std::string delete_from_to_sql(const dynamic::DeleteFrom& _stmt) noexcept;
std::string drop_to_sql(const dynamic::Drop& _stmt) noexcept;
std::string insert_to_sql(const dynamic::Insert& _stmt) noexcept;
template <class InsertOrWrite>
std::string insert_or_write_to_sql(const InsertOrWrite& _stmt) noexcept;
std::string properties_to_sql(const dynamic::types::Properties& _p) noexcept;
@@ -219,7 +220,8 @@ std::string drop_to_sql(const dynamic::Drop& _stmt) noexcept {
return stream.str();
}
std::string insert_to_sql(const dynamic::Insert& _stmt) noexcept {
template <class InsertOrWrite>
std::string insert_or_write_to_sql(const InsertOrWrite& _stmt) noexcept {
using namespace std::ranges::views;
const auto in_quotes = [](const std::string& _str) -> std::string {
@@ -314,7 +316,7 @@ std::string to_sql_impl(const dynamic::Statement& _stmt) noexcept {
return drop_to_sql(_s);
} else if constexpr (std::is_same_v<S, dynamic::Insert>) {
return insert_to_sql(_s);
return insert_or_write_to_sql(_s);
} else if constexpr (std::is_same_v<S, dynamic::SelectFrom>) {
return select_from_to_sql(_s);
@@ -322,6 +324,9 @@ std::string to_sql_impl(const dynamic::Statement& _stmt) noexcept {
} else if constexpr (std::is_same_v<S, dynamic::Update>) {
return update_to_sql(_s);
} else if constexpr (std::is_same_v<S, dynamic::Write>) {
return insert_or_write_to_sql(_s);
} else {
static_assert(rfl::always_false_v<S>, "Unsupported type.");
}

49
tests/postgres/test_insert_and_read.cpp Normal file
View File

@@ -0,0 +1,49 @@
#include <gtest/gtest.h>
#include <rfl.hpp>
#include <rfl/json.hpp>
#include <sqlgen.hpp>
#include <sqlgen/postgres.hpp>
#include <vector>
namespace test_insert_and_read {
struct Person {
sqlgen::PrimaryKey<uint32_t> id;
std::string first_name;
std::string last_name;
int age;
};
TEST(postgres, test_insert_and_read) {
const auto people1 = std::vector<Person>(
{Person{
.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10},
Person{.id = 2, .first_name = "Lisa", .last_name = "Simpson", .age = 8},
Person{
.id = 3, .first_name = "Maggie", .last_name = "Simpson", .age = 0}});
const auto credentials = sqlgen::postgres::Credentials{.user = "postgres",
.password = "password",
.host = "localhost",
.dbname = "postgres"};
using namespace sqlgen;
const auto people2 = sqlgen::postgres::connect(credentials)
.and_then(drop<Person> | if_exists)
.and_then(begin_transaction)
.and_then(create_table<Person> | if_not_exists)
.and_then(insert(std::ref(people1)))
.and_then(commit)
.and_then(sqlgen::read<std::vector<Person>>)
.value();
const auto json1 = rfl::json::write(people1);
const auto json2 = rfl::json::write(people2);
EXPECT_EQ(json1, json2);
}
} // namespace test_insert_and_read

62
tests/postgres/test_insert_and_read_two_tables.cpp Normal file
View File

@@ -0,0 +1,62 @@
#include <gtest/gtest.h>
#include <rfl.hpp>
#include <rfl/json.hpp>
#include <sqlgen.hpp>
#include <sqlgen/postgres.hpp>
#include <vector>
namespace test_insert_and_read_two_tables {
struct Person {
sqlgen::PrimaryKey<uint32_t> id;
std::string first_name;
std::string last_name;
int age;
};
struct Children {
uint32_t id_parent;
sqlgen::PrimaryKey<uint32_t> id_child;
};
TEST(postgres, test_insert_and_read_two_tables) {
const auto people1 = std::vector<Person>(
{Person{
.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10},
Person{.id = 2, .first_name = "Lisa", .last_name = "Simpson", .age = 8},
Person{
.id = 3, .first_name = "Maggie", .last_name = "Simpson", .age = 0}});
const auto children =
std::vector<Children>({Children{.id_parent = 0, .id_child = 1},
Children{.id_parent = 0, .id_child = 2},
Children{.id_parent = 0, .id_child = 3}});
const auto credentials = sqlgen::postgres::Credentials{.user = "postgres",
.password = "password",
.host = "localhost",
.dbname = "postgres"};
using namespace sqlgen;
const auto people2 = sqlgen::postgres::connect(credentials)
.and_then(drop<Person> | if_exists)
.and_then(drop<Children> | if_exists)
.and_then(begin_transaction)
.and_then(create_table<Person> | if_not_exists)
.and_then(create_table<Children> | if_not_exists)
.and_then(insert(std::ref(people1)))
.and_then(insert(std::ref(children)))
.and_then(commit)
.and_then(sqlgen::read<std::vector<Person>>)
.value();
const auto json1 = rfl::json::write(people1);
const auto json2 = rfl::json::write(people2);
EXPECT_EQ(json1, json2);
}
} // namespace test_insert_and_read_two_tables

3
tests/postgres/test_insert_dry.cpp
View File

@@ -16,8 +16,7 @@ TEST(postgres, test_insert_dry) {
const auto query = sqlgen::Insert<TestTable>{};
const auto expected =
"COPY \"public\".\"TestTable\"(\"field1\", \"field2\", \"id\", "
"\"nullable\") FROM STDIN WITH DELIMITER '\t' NULL '\e' CSV QUOTE '\a';";
R"(INSERT INTO "TestTable" ("field1", "field2", "id", "nullable") VALUES ($1, $2, $3, $4);)";
EXPECT_EQ(sqlgen::postgres::to_sql(query), expected);
}

50
tests/postgres/test_write_and_read_curried.cpp Normal file
View File

@@ -0,0 +1,50 @@
#ifndef SQLGEN_BUILD_DRY_TESTS_ONLY
#include <gtest/gtest.h>
#include <rfl.hpp>
#include <rfl/json.hpp>
#include <sqlgen.hpp>
#include <sqlgen/postgres.hpp>
#include <vector>
namespace test_write_and_read_curried {
struct Person {
sqlgen::PrimaryKey<uint32_t> id;
std::string first_name;
std::string last_name;
int age;
};
TEST(postgres, test_write_and_read_curried) {
const auto people1 = std::vector<Person>(
{Person{
.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10},
Person{.id = 2, .first_name = "Lisa", .last_name = "Simpson", .age = 8},
Person{
.id = 3, .first_name = "Maggie", .last_name = "Simpson", .age = 0}});
const auto credentials = sqlgen::postgres::Credentials{.user = "postgres",
.password = "password",
.host = "localhost",
.dbname = "postgres"};
using namespace sqlgen;
const auto people2 = postgres::connect(credentials)
.and_then(drop<Person> | if_exists)
.and_then(write(std::ref(people1)))
.and_then(sqlgen::read<std::vector<Person>>)
.value();
const auto json1 = rfl::json::write(people1);
const auto json2 = rfl::json::write(people2);
EXPECT_EQ(json1, json2);
}
} // namespace test_write_and_read_curried
#endif

43
tests/sqlite/test_insert_and_read.cpp Normal file
View File

@@ -0,0 +1,43 @@
#include <gtest/gtest.h>
#include <rfl.hpp>
#include <rfl/json.hpp>
#include <sqlgen.hpp>
#include <sqlgen/sqlite.hpp>
#include <vector>
namespace test_insert_and_read {
struct Person {
sqlgen::PrimaryKey<uint32_t> id;
std::string first_name;
std::string last_name;
int age;
};
TEST(sqlite, test_insert_and_read) {
const auto people1 = std::vector<Person>(
{Person{
.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10},
Person{.id = 2, .first_name = "Lisa", .last_name = "Simpson", .age = 8},
Person{
.id = 3, .first_name = "Maggie", .last_name = "Simpson", .age = 0}});
using namespace sqlgen;
const auto people2 = sqlite::connect()
.and_then(begin_transaction)
.and_then(create_table<Person> | if_not_exists)
.and_then(insert(people1))
.and_then(commit)
.and_then(sqlgen::read<std::vector<Person>>)
.value();
const auto json1 = rfl::json::write(people1);
const auto json2 = rfl::json::write(people2);
EXPECT_EQ(json1, json2);
}
} // namespace test_insert_and_read

43
tests/sqlite/test_insert_by_ref_and_read.cpp Normal file
View File

@@ -0,0 +1,43 @@
#include <gtest/gtest.h>
#include <rfl.hpp>
#include <rfl/json.hpp>
#include <sqlgen.hpp>
#include <sqlgen/sqlite.hpp>
#include <vector>
namespace test_insert_by_ref_and_read {
struct Person {
sqlgen::PrimaryKey<uint32_t> id;
std::string first_name;
std::string last_name;
int age;
};
TEST(sqlite, test_insert_by_ref_and_read) {
const auto people1 = std::vector<Person>(
{Person{
.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10},
Person{.id = 2, .first_name = "Lisa", .last_name = "Simpson", .age = 8},
Person{
.id = 3, .first_name = "Maggie", .last_name = "Simpson", .age = 0}});
using namespace sqlgen;
const auto people2 = sqlite::connect()
.and_then(begin_transaction)
.and_then(create_table<Person> | if_not_exists)
.and_then(insert(std::ref(people1)))
.and_then(commit)
.and_then(sqlgen::read<std::vector<Person>>)
.value();
const auto json1 = rfl::json::write(people1);
const auto json2 = rfl::json::write(people2);
EXPECT_EQ(json1, json2);
}
} // namespace test_insert_by_ref_and_read

7
tests/sqlite/test_to_insert.cpp
View File

@@ -1,8 +1,9 @@
#include <gtest/gtest.h>
#include <sqlgen.hpp>
#include <sqlgen/dynamic/Insert.hpp>
#include <sqlgen/sqlite.hpp>
#include <sqlgen/transpilation/to_insert.hpp>
#include <sqlgen/transpilation/to_insert_or_write.hpp>
namespace test_to_insert {
@@ -14,7 +15,9 @@ struct TestTable {
};
TEST(sqlite, test_to_insert) {
const auto insert_stmt = sqlgen::transpilation::to_insert<TestTable>();
const auto insert_stmt =
sqlgen::transpilation::to_insert_or_write<TestTable,
sqlgen::dynamic::Insert>();
const auto conn = sqlgen::sqlite::connect().value();
const auto expected =
R"(INSERT INTO "TestTable" ("field1", "field2", "id", "nullable") VALUES (?, ?, ?, ?);)";

41
tests/sqlite/test_write_and_read_curried.cpp Normal file
View File

@@ -0,0 +1,41 @@
#include <gtest/gtest.h>
#include <functional>
#include <rfl.hpp>
#include <rfl/json.hpp>
#include <sqlgen.hpp>
#include <sqlgen/sqlite.hpp>
#include <vector>
namespace test_write_and_read_curried {
struct Person {
sqlgen::PrimaryKey<uint32_t> id;
std::string first_name;
std::string last_name;
int age;
};
TEST(sqlite, test_write_and_read_curried) {
const auto people1 = std::vector<Person>(
{Person{
.id = 0, .first_name = "Homer", .last_name = "Simpson", .age = 45},
Person{.id = 1, .first_name = "Bart", .last_name = "Simpson", .age = 10},
Person{.id = 2, .first_name = "Lisa", .last_name = "Simpson", .age = 8},
Person{
.id = 3, .first_name = "Maggie", .last_name = "Simpson", .age = 0}});
using namespace sqlgen;
const auto people2 = sqlite::connect()
.and_then(write(std::ref(people1)))
.and_then(sqlgen::read<std::vector<Person>>)
.value();
const auto json1 = rfl::json::write(people1);
const auto json2 = rfl::json::write(people2);
EXPECT_EQ(json1, json2);
}
} // namespace test_write_and_read_curried