Commit Graph

96 Commits

Author SHA1 Message Date
silverqx
160375a179 macros added no_sanitize attribute 2024-11-22 22:10:25 +01:00
silverqx
e24064b7fd qmake moved all feature to private/ folder
Moved all features which aren't used through CONFIG+=xyz and are loaded
using the load(private/xyz) function to the qmake/features/private/
folder.

Currently, the qmake/features/ folder contains only features which
should be used using the CONFIG+= qmake variable.
2024-06-25 13:44:52 +02:00
silverqx
039e9fcf4b drivers get rid of Drivers::SqlError 🔥🚀🥳
This is part 2 of: used TinyDrivers exceptions everywhere.
The Drivers::SqlError was dropped and renamed to DummySqlError which
does nothing and it exists only for API compatibility with QtSql module
and it also helps to avoid #ifdef-s in users/client code.

There are 3 exceptions (what is all this about) and they are SqlError,
QueryError, and SqlTransactionError. The SqlError is a general SQL
exception class that is thrown eg. during connecting to SQL server
or during all other SQL API calls that are not database queries or
transaction queries.

The second part of this commit was to modify all error handling
in TinyOrm library and adapt it to exceptions thrown from
the TinyDrivers library.
Many #ifdef-s were added because of this because QtSql module returns
bool and then you need to obtain the QSqlError and TinyDrivers library
throws exceptions.

This commit was huge, also it's very weird and this error handling
things are unpopular and everybody has different opinion on this, but
it is what it is.

There is not way I would return bool and set error state instead of
exceptions in TinyDrivers, never.
Also, there is no way I would catch Drivers::SqlError/QueryError/
SqlTransactionError in TinyOrm library and translate these TinyDrivers
exceptions to TinyOrm exceptions with the same names to throw only one
set of exceptions, no way. TinyDrivers throws its own exceptions and
TinyOrm linked against QtSql throws its own exceptions (with the same
names but in different namespace).

 - removed all setLastError() and throw exceptions instead
 - added new constants
 - optimized #include-s
 - changed the causedByLostConnection() method signature
 - enhanced the ManagesTransactions class
 - extracted code to the mysqlClose()
 - removed MySqlDriverPrivate::q_ptr (not needed anymore)
 - removed createError() and createStmtError() (not needed anymore) and
   added prepareMySqlError() and prepareStmtError() as counterparts
 - renamed method to errorMessageForStmtFetch()
 - extracted the replaceBindingsInSql() because it's used from both
   TinyOrm and TinyDrivers libraries
 - enhanced replaceBindingsInSql()
   - added support for BLOB-s
 - the SqlError, QueryError, and SqlTransactionError are excluded from
   TinyOrm library build and include path if compiled against
   the TinyDrivers library (they are not needed and used anymore)
 - added connection name to SqlResult class, it's used in exception
   messages
 - also updated auto tests

bugfix-es:

 - qmake propagate the TINYDRIVERS_MYSQL_DRIVER DEFINE (made it public),
   cmake build is OK
 - removed mysql_thread_init()
 - changed call order for the mysql_thread_end()
2024-04-07 18:12:00 +02:00
silverqx
247bb5e936 tom added TinyDrivers to tom about
- and also to the tom --version
 - added components to Dependencies section
 - prints info also about TinyMySql
 - takes into account all build types static/shared/loadable
 - tests both build systems qmake/CMake
2024-04-07 18:11:56 +02:00
silverqx
59eedcfac3 drivers prepared for swappable Qt/Tiny SQL drivers
Added sqldrivermappings.hpp for swappable Qt/Tiny SQL drivers.
2024-04-07 18:11:35 +02:00
silverqx
09911ffc6b revisited static_cast<> for integral types 2024-03-26 21:02:44 +01:00
silverqx
5a661c1b93 extracted stringify C macro to own hpp file 2024-02-05 16:27:49 +01:00
silverqx
5867331647 qmake initialize empty variables 2023-12-04 18:18:18 +01:00
silverqx
60b4fbc914 qmake changed to *= for all INCLUDEPATH
Changed all to INCLUDEPATH *= $$PWD.
2023-09-09 09:13:32 +02:00
silverqx
0bb7a9c331 renamed exceptions 2023-07-11 17:37:28 +02:00
silverqx
411a635e6c added Appending Values To JSON support 🕺
These Appends are appended to the serialized models during the toMap(),
toVector(), or toJson() methods call.

Appends names are and must be mapped using the u_mutators hash map
(in the same way like u_relations) because c++ doesn't support advanced
reflection.

Appends are mapped to methods (can be protected) that return
the Casts::Attribute. This Casts::Attribute is a wrapper to return
a new attribute value.

Appends accept u_visible and u_hidden.

For the Appends that return QDate or QDateTime are the counterpart
serializeDate() or serializeDateTime() methods called.

 - added unit tests
2023-07-11 17:14:20 +02:00
silverqx
d4444e4c28 added Hiding Attributes support
A user can define the u_visible and u_hidden std::set static data
member for visible and hidden attributes on his models and they will be
used during serialization.

The relationships and also u_snakeAttributes are correctly taken into
account.

 - added unit tests
 - added example to Models::Torrent
2023-07-11 08:31:06 +02:00
silverqx
db5e243db7 extracted relation stores to own files
The big HasRelationStore class with nested classes is finally divided
into smaller pieces, every store is in its own file in the Support NS.

Performance is the same as the old code, thanks god. 🙏

mt new:  564
mt old:  565
st new:  959 294
st old:  959 305
2023-07-01 10:23:05 +02:00
silverqx
7f344a2145 added ModelsCollection 🔥🚀🎉
The Orm::Tiny::Types::ModelsCollection is container class with
additional handy methods like pluck(), whereIn(), filter(), ...
It extends the QVector. The template parameter can be the Model or
Model *, the model must be the Derived model type eg. Torrent, Post, ...

All ORM-related methods return the ModelsCollection so it can be easily
filtered, changed, transformed with one simple line or callback.
All the algorithms are nicely encapsulated in the ModelsCollection
methods.

Currently, there is this pattern, all TinyBuilder or Model methods are
returning the ModelsCollection<Derived> and all relation related
methods are returning the ModelsCollection<Derived *>.

 - added Album and AlbumImage migrations and seeder data (also PHP)
 - added Album and AlbumImage models
 - added functional tests for ModelsCollection<Model> and
   ModelsCollection<Model *>
 - all ModelsCollection methods or algorithms are unit tested
 - replaced all occurences of the QVector<Model> and QVector<Model *>
   with the ModelsCollection<Model> and ModelsCollection<Model *> 😵‍💫
 - tests added Model::findMany()
 - added reserve() on few places
 - removed useless #include-s
2023-05-05 13:06:15 +02:00
silverqx
f545316b06 added Orm::Utils::NotNull<T>
Provides the same functionality like the gsl::not_null<T>.

 - also provides the Orm::Utils::makeNotNull() factory global function
2023-03-30 10:41:25 +02:00
silverqx
5443e1260e added LostConnectionError exception 2023-03-14 12:48:25 +01:00
silverqx
b13728db08 refactored PostgreSQL search_path configuration
Changed from the 'schema' configuration option to the 'search_path',
the 'schema' config. option was dropped and currently throws exception
with a suggestion to use the 'search_path' instead.

The new 'search_path' config. option can be the QString or QStringList
and also supports to set an empty search_path using '' or empty QString
or QStringList.
This search_path config. option is correctly parsed and quoted.
Setting the search_path is initiated from the PostgresConnector
right after the database connection is established.

The PostgresSchemaBuilder is actively using the search_path during
practically all method calls (dropAllTables(), getColumnListing(),
hasTable(), ...). If the search_path is not defined then it queries
the database for the real search_path because the search_path can be
changed anytime and the search_path configuration option value can be
incorrect (out of sync).
It also works correctly with the empty search_path, in this case a fully
qualified table names are required. If the table name is not fully
qualified then the SearchPathEmptyError exception is thrown.
And it also works correctly with the "$user" PostgreSQL's variable, this
variable is resolved to the config["username"] during obtaining
the search_path from the database.

In the pretend mode it behaves little different, if pretending then
the 'search_path' is obtained from the connection configuration and
if the 'search_path' is not defined in the config. the 'public' will
be used as the default.

 - added many new unit and functional tests which are testing all the
   above described scenarios
   - added new functional test case tst_PostgreSQL_SchemaBuilder_f
   - added new functional test case tst_SchemaBuilder
   - added new tests for dropAllTable(), dropAllViews(), hasTable(),
     getColumnListing(), createDatabase(), dropDatabaseIfExists()
     for ALL supported databases
 - added new search_path constant
 - renamed env. variable to DB_PGSQL_SEARCHPATH (from DB_PGSQL_SCHEMA)
 - added new search_path and dont_drop configuration validations to
   check correct type (QString or QStringList)
 - added new 'schema' config. option validation
 - added new exception SearchPathEmptyError
 - started the TinyUtils::Databases refactor
2023-02-11 09:39:28 +01:00
silverqx
6b20ce118d added SQLiteDatabaseDoesNotExistError exception
Used it instead of RuntimeError exception.
2023-02-01 18:52:18 +01:00
silverqx
9527aa4f05 refactored configuration parsing
- extracted logic to own classes
 - every driver has it's own class and can have driver-specific logic
 - the 'options' option has it's own parser class, is declared as
   the mixin class
2023-02-01 09:55:42 +01:00
silverqx
3bb75d13d1 extracted all null QVariant-s to library class
It helps to avoid the #ifdef-s all over the code.
2022-10-30 12:06:01 +01:00
silverqx
1ded27bbc8 QDateTime overhaul 🤯🤐🙃
Fixed buggy behavior of QDateTime values during SELECT, INSERT, and
UPDATE queries for all supported QtSql drivers. Behavior is fixed if
querying the database using Orm::QueryBuilder or TinyBuilder/Model.
Can't be fixed if using raw queries using the QSqlQuery because I don't
have any control over this code.

Every QtSql driver behaves differently in how it works with
the QDateTime and datetime-related database types. It doesn't have only
one problem, it has various kinds of problems, eg. it returns all
QDateTime objects in the local time zone. All issues are summarized
in NOTES.TXT under "QDateTime and database date/time types:"
section.

This buggy behavior can be fixed or corrected using a new "qt_timezone"
database connection configuration option. It accepts the time zone value
in various formats (QTimeZone, Qt::TimeSpec, int number as an offset
from UTC, QString eg. +02:00, Europe/Prague or Orm::QtTimeZoneConfig).
This "qt_timezone" value affects how the  QueryBuilder and TinyBuilder
send and receives datetime-related types to/from database. It should be
set to the same time zone value as the "timezone" connection
configuration option.

During the INSERT and UPDATE statements, it CONVERTS QDateTime's
time zone to the qt_timezone value before the statement is sent
to the database.
And during the SELECT statements it SETS QDateTime's time zone value
after the values are obtained from the database.

So if you set the "timezone" to UTC and "qt_timezone" to eg. Qt::UTC or
QTimeZone::utc(), then QDateTime values will have set the correct
time zone, in this case, the time zone will be UTC.

For the SQLite database was also added the "return_qdatetime" connection
configuration option which default value is true. By default,
the QSQLITE driver returns datetime values as QString.
The "return_qdatetime" controls this behavior and if is set to true then
the QDateTime will be returned instead.

TinyORM QueryBuilder returns the Orm::SqlQuery instead of QSqlQuery
from methods for which the QDateTime's time zone should be corrected.
That are eg. the select(), selectOne(), unprepared(), or chunk(),
chunkById(), each(), eachById(), ...

Orm::SqlQuery is a simple wrapper around the QSqlQuery whose
responsibility is only to fix a QDateTime's time zone.

Also unified the QDate behavior across all QtSql drivers.

 - added a new migration for the datetime table for testing QDateTime
   and datetime-related database types
 - added new Datetime model
 - added functional tests for testing datetime-related queries
   - testing QDateTime for all supported drivers
   - testing with different time zones, UTC, +02:00
   - testing QDate for all supported drivers
 - added two new connection configuration options "qt_timezone" and
   "return_qdatetime" which is for the SQLite database only
 - added returnQDateTime()/setReturnQDateTime() getter/setter
   to the SQLiteConnection class
 - added getQtTimeZone()/setQtTimeZone()/isConvertingTimeZone()
   to the DatabaseConnection class
 - tests, fixed all QDateTime instances, changed time zone to UTC, so
   auto tests are now UTC, they force also MySQL and PostgreSQL server
   time zone session variable to UTC

Others:

 - StringUtils moved from orm/tiny/utils/ to the orm/utils/ folder
   to the ::Orm::Utils namespace
 - enhanced all database connection constructors, used rvalue references
 - added delegated DatabaseConnection() constructor
   for SQLiteConnection() because of m_returnQDateTime
2022-10-27 14:30:00 +02:00
silverqx
61bd9e87f1 added Soft Deleting support 🤯🙌
In addition to actually removing records from your database, TinyORM
can also "soft delete" models. When models are soft deleted, they are
not actually removed from your database. Instead, a deleted_at attribute
is set on the model indicating the date and time at which the model was
"deleted".

 - added docs
 - added all proxies
 - added complete functional tests and a few unit tests
 - carefully verified all possible scenarios because it changes
   practically all queries 🤯

Others:

 - also added from() to all proxies
 - bugfix in seeders, bad ID in the PostgreSQL primary key (sequence)
2022-08-26 18:21:45 +02:00
silverqx
96b80488d0 added HasOne/BelongsTo::is/isNot
It allows comparing a related model without issuing a query to retrieve
that model.

 - added functional tests for both HasOne and also BelongsTo relations
 - added docs

Others:

 - added the new unit test case tst_Relations_Connection_Independent
2022-08-19 16:31:38 +02:00
silverqx
5ff160dec5 use upsert alias on MySQL
The MySQL >=8.0.19 supports aliases in the VALUES and SET clauses
of INSERT INTO ... ON DUPLICATE KEY UPDATE statement for the row to be
inserted and its columns. It also generates warning from >=8.0.20 if
an old style used.

This enhancement detects the MySQL version and on the base of it uses
alias during the upsert call.

Also a user can override the version through the MySQL database
configuration. It helps to save/avoid one database query
(select version()) during the upsert method call or during connecting
to the MySQL database (version is needed if strict mode is enabled).

 - added unit and functional tests
 - updated number of unit tests to 1402
 - updated upsert docs
 - added ConfigUtils to avoid duplicates

Others:

 - added the version database configuration everywhere
 - docs added few lines about MySQL version configuration option
 - docs updated database configurations, added a new missing options
2022-08-12 15:37:56 +02:00
silverqx
53eb512d2c added Concerns::InteractsWithPivotTable 🤯
Move all pivot related methods for the BelongsToMany relation into
the new InteractsWithPivotTable concern.

 - rvalue overloads for BelongsToMany::as/withPivot
 - added reserve where possible
 - whitespaces and formatting
 - also complete revisit
2022-08-03 13:51:23 +02:00
silverqx
6f21c15794 added BuildsQueries concern for TinyBuilder
Added chunk, each, chunkMap, chunkById, eachById, sole, tap
in TinyBuilder::Concerns::BuildsQueries and TinyBuilder::soleValue().

 - added tests
 - added proxies to Model and Relation
 - also added missing proxy for forPageAfterId()
2022-07-28 19:38:29 +02:00
silverqx
2557f66594 added BuildsQueries concerns
Added chunk, each, chunkById, eachById, sole, tap in BuildsQueries and
QueryBuilder::soleValue().

 - added tests
 - added docs
2022-07-27 08:38:02 +02:00
silverqx
67dfa12b36 added tap helper
It calls the given callback with the given value then return the value.

Also used it on few places as a proof that it works, the old code
looks cleaner though because it's flat.
2022-06-02 14:57:35 +02:00
silverqx
0bcbcd2431 added support for PostgreSQL schema builder 🎉👌
- added a new config. option dont_drop, allows to define table that
   will be excluded during dropAllTables(), has a default value
   spatial_ref_sys for PostGIS
 - implemented fluent commands, used only by the PostgreSQL Comment
   command
 - added tests for PostgreSQL schema builder
 - reworked tst_Migrate to be able to run on all supported connections,
   currently PostgreSQL and MySQL
 - updated docs

Unrelated:

 - added a new reference class IndexDefinitionReference for a nice API
   during index definition, contains algorithm and language
 - unified selectFromWriteConnection() in schema builders
2022-05-23 10:14:25 +02:00
silverqx
1b3b431873 extracted Model unguarded to own class
Extracting these guard related methods to own class allows to call
GuardedModel::unguarded() regardless of a template parameters needed
on the Model class, before was needed to call
Model<Torrent, Relations..>::unguarded().

 - also made the g_unguarded atomic
 - bugfix reguard() try-catch-finally in GuardedModel::unguarded()
2022-05-16 14:25:24 +02:00
silverqx
a6213a6a9d added migrations 🔥🚀
Almost everything is implemented, possible to migrate up, down
(rollback), migrate by steps (works for up/down too), reset, refresh,
fresh, wipe database, showing status, installing migration table.

Command line interface is superb, it supports ansi coloring, verbosity,
no-interactive mode, force option, env option to select current env.

It has enhanced ansi coloring (vt100 terminal) detection, when ansi or
no-ansi option is not passed it can detect whether terminal supports
coloring.
Ansi coloring is disabled when redirection to file is detected (can
be overridden by ansi/no-ansi options).
Supports NO_COLOR env. variable (https://no-color.org/) and can detect
the ConEmu, Hyper, and xterm on Windows.

Carefully implemented help and list commands, list command can print
supported commands by namespace.

Advanced make migration command offers great command interface for
creating migration classes, supports options for generating empty,
create, or update migration classes.

Unfinished make project command, will support creating qmake, qmake
subproject, and cmake, cmake subproject projects. Later will be
extracted to own executable tomproject.exe for rapidly generating a new
TinyORM projects.

Other implemented commands are env that prints current env. and inspire
that displays an inspiring quote 😁.

Verbose supports 5 levels quiet, normal, verbose, very verbose, and
debug.

Possibility to disable compilation of the tom command related code using
TINYORM_DISABLE_TOM c macro, for the qmake exists disable_tom CONFIG
option and for the CMake exist TOM configuration option.

Confirmable interface that ask Y/N confirmation during migrate when
env. == production, can be overridden by --force option.

Whole tom terminal application supports or is implemented with UTF-8
support, also implemented UTF-16 output methods but they are not needed.
Input also supports UTF-8, currently only Y/N input is needed by the
Confirmation concern.

All migrate commands also support the --pretend option and the --env
option, when env. is production then tom asks confirmation to migrate,
it can be overridden by the --force option.

Added the tom example project, it is a complete command-line migration
application, it uses migrations from the tests.

Implementing this was really fun 🙃😎.

 - added 14 functional tests to test migrations up/down, stepping,
   installing migration table, refresh, reset on MySQL database
 - added unit test to check version number in tom.exe executable
 - new tom exception classes
 - created dozens of a new todo tasks 😂🤪, currently 348 todos 😎
 - added some info messages to the qmake build about building features
 - in the debug build and with the -vvv option enable debugging of sql
   queries
 - enhanced RC and manifest file generation, allowed to pass a custom
   basename for a RC/manifest file as the 3. argument and a custom
   replace token for the CMake genex as the 4. argument
 - bugfix disabled #pragma code_page(65001) // UTF-8 in RC files, it
   messes up the © character

Output of tom exe without arguments and options:

Common options:
      --ansi            Force ANSI output
      --no-ansi         Disable ANSI output
      --env             The environment the command should run under
  -h, --help            Display help for the given command. When no
                        command is given display help for the list
                        command
  -n, --no-interaction  Do not ask any interactive question
  -q, --quiet           Do not output any message
  -V, --version         Display this application version
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal
                        output, 2 for more verbose output and
                        3 for debug

Available commands:
  env                   Display the current framework environment
  help                  Display help for a command
  inspire               Display an inspiring quote
  list                  List commands
  migrate               Run the database migrations
 db
  db:wipe               Drop all tables, views, and types
 make
  make:migration        Create a new migration file
  make:project          Create a new Tom application project
 migrate
  migrate:fresh         Drop all tables and re-run all migrations
  migrate:install       Create the migration repository
  migrate:refresh       Rollback and re-run all migrations
  migrate:reset         Rollback all database migrations
  migrate:rollback      Rollback the last database migration
  migrate:status        Show the status of each migration
2022-04-20 15:45:35 +02:00
silverqx
a2714be30e added schema builder 🔥🚀
Added practically everything I wanted to have in except for updating
columns.
Needed to name the schema namespace as Orm::SchemaNs to avoid collision
with the Orm::Schema class.

 - unit tests with great coverage
 - new schema constants
 - a new prefix_indexes and engine DB conncetion configurations

Others:

 - IsModel tag
 - enhanced columnize in the BaseGrammar
 - used a new columnize logic in all grammars
 - new constants
 - new container utils class for joining containers
 - new DB::driver() getter for QSqlDriver
 - avoid possible crash in tests with pretend, added empty log checks
 - clang tidy, excluded to word from short variable names
2022-03-22 09:53:33 +01:00
silverqx
5956cb86c5 added OrmError base class for all exceptions 2022-02-14 17:23:31 +01:00
silverqx
8794b8b285 extracted relation types to own header file 2022-01-16 18:24:05 +01:00
silverqx
bbfca1594e extracted transactions to ManagesTransactions
Transactions extracted to the Concerns::ManagesTransactions base class.

 - bugfix includes
 - logConnected()/logDisconnected() code wrapped in TINYORM_MYSQL_PING
 - hitTransactionalCounters() extracted to Concerns::CountsQueries
 - convertNamedToPositionalBindings() extracted to Concerns::LogsQueries
2022-01-14 21:26:02 +01:00
silverqx
ff0697e1a5 extracted logging queries to LogQueries
Logging queries extracted to the Concerns::LogsQueries base class.
2022-01-14 17:37:58 +01:00
silverqx
b36888ae93 extracted queries counters to CountsQueries
Queries counters extracted to the Concerns::CountsQueries base class.

 - removed ConnectionInterface 😕😎🙌
 - all methods are returning DatabaseConnection & instead of
   ConnectionInterface
 - includes cleanup after extraction
2022-01-14 16:04:17 +01:00
silverqx
280713bcd1 ConnectionOverride moved to own tiny types ns 2022-01-07 13:07:40 +01:00
silverqx
b092191aa5 concepts divided to orm and tiny concepts 2022-01-07 12:54:34 +01:00
silverqx
dbc375f1fe added HasRelationships concerns
Extracted appropriate methods from the Tiny::Model class to own
HasRelationships concerns.
2022-01-07 12:35:47 +01:00
silverqx
5a593224fe extracted CRTP static cast to own macro files
Added to macro files, one with derived model cast and another with
derived model cast and with cast to the base model.
2022-01-06 11:14:48 +01:00
silverqx
7a975f7b99 added HasAttributes and HasTimestamps concerns
Extracted appropriate methods from the Tiny::Model class to own
HasAttributes and HasTimestamps concerns.
2022-01-05 21:02:57 +01:00
silverqx
657f39b104 moved SyncChanges to Tiny::Types 2022-01-03 13:11:01 +01:00
silverqx
087bf68999 moved LOG_EXECUTED_QUERY macro to own file 2022-01-01 16:31:59 +01:00
silverqx
92775f444c divided Orm::Utils
While ORM can be disabled, Orm::Utils has been split into Orm::Utils
and Orm::Tiny::Utils (Orm::Tiny::TinyUtils alias).
So when ORM is disabled then utils used only in the ORM will be
excluded from a final build.

 - bugfix cmake undefined ORM option in TinySources
2022-01-01 14:28:42 +01:00
silverqx
fc477bf1e5 added qmake/cmake option to disable ORM
Added TINYORM_DISABLE_ORM macro controlled by ORM for CMake and
disable_orm for qmake. When TINYORM_DISABLE_ORM macro is defined then
only the query builder without ORM is compiled.
Also excludes ORM-related unit tests.
2021-12-31 11:40:12 +01:00
silverqx
dcbb113fc8 added Fs::resolveHome()
It resolves ~ home prefix to the full filepath
2021-11-21 19:30:51 +01:00
silverqx
037207cd42 moved DatabaseConfiguration to support/ 2021-11-13 19:09:15 +01:00
silverqx
57da4a109e renamed macros.hpp to likely.hpp
It contained T_LIKELY/T_UNLIKELY only.
2021-11-12 20:40:02 +01:00
silverqx
b39b4dcf48 fixed errors for Linux clang/gcc 2021-11-12 12:27:03 +01:00