This commit does a lot of refactoring. But most noticeably it changes
two things:
1) Instead of saving all objects (tables, views, indices, triggers) of a
schema in a common map, we now store tables/views, indices and
triggers in three separate maps. This has a number of benefits:
- It resembles more closely how SQLite stores its data internally and
therefore achieves greater compatability e.g. for databases with a
view and a trigger with the same name.
- It reduces the need for runtime polymorphism. This makes the code
run a bit faster.
- By working with explicit types more often more error checking can
be done at compile time, making the code less error prone.
- The code becomes a lot clearer to read.
2) By making View inherit form Table, views are now a sort of tables.
This has the following benefits:
- This is a again how SQLite stores views internally which again
should increase compatibility a bit.
- We mostly treat views and tables the same anyway and with these
changes we can unify the code for them even more.
Change some function parameters which are named like "dont..." to make a
positive statement. This should hopefully avoid some confisions with
double negation.
The feature is implemented using the SQLite backup API.
https://sqlite.org/backup.html
This allows saving in-memory databases and saving database files to another
file name.
When modifying the value of a field in the Browse Data tab, we used to
always hand over the new value to SQLite as text. This works most of the
time but can cause problems e.g. when a CHECK constraint checks the data
type of the value and expects something other than text. It is also a
pretty lazy approach in general.
This commit checks if the new value matches the affinity data type of
the column, and if it does hands over the new value as the correct data
type.
See issue #1952.
Instead of registering the error log callback function every time a
database file is opened, we now register it only once when the
application is first started. This way we can avoid calling
sqlite_shutdown which can have unintended side effects.
See issue #1932.
This add initial and mostly untested support for WITHOUT ROWID tables
with multiple primary key columns. It should now be possible to update
and to delete records in these tables.
This commit also improves the overall handling of multiple primary key
columns in preparation for better support of them in general.
Note that this makes us depend on an SQLite version with a built-in JSON
extension.
See issues #516, #1075, and #1834.
Add a new logging section to the SQL log panel called "Error Log".
Register a callback to receive errors and warnings from SQLite and add
them to the new error log section.
See issue #1754.
Allow user to define their own display formats. The SQL code is editable
and the combo box changes automatically to custom if the user edits one
of the predefined formats.
The display format (either custom or predefined) is validated with a
case-insensitive regex so at least it contains a function applied to the column
name.
Added a new callback for executeSQL following the model of sqlite3_exec.
Using this, the number of columns is got from a checking query execution. If
it is not one, the custom display format is also rejected.
Note that these validations might still be fooled in some unforeseen way, but
the users should know what they're doing.
See issue #573
This commit attempts to speed up the SQL import a bit by removing
unnecessary conversions and counting of characters. It's still far from
perfect though.
See issue #1045.
In the Edit Table dialog we used to call our alterTable function (which
works around SQLite's missing full ALTER TABLE support by - besided
other things - copying all the data of the table) for pretty much every
change immediately. This was taking a lot of time for larger tables.
Our alterTable function allowed any number of changes if they affect
only one field of the table at once. So we could have reduced the number
of calls a lot by just using that capability. Instead however, this
commit improves the alterTable function to make possible transforming a
table completely in just one call. It does so by taking the new table
schema and using that without further modification. It also takes a new
parameter to keep track of what column in the old table becomes what
column in the new table, so the data can be preserved.
This commit obviously also changes the Edit Table dialog to make proper
use of the new features. This means that whatever changes you make to a
table, you will only have to wait once until for the alterTable call,
and that's when clicking the OK button.
See issue #1444.
waitForDbRelease receives a parameter for waiting, cancelling the other
operation or asking the user; defaulting to ask.
The wait option is used for the conditional formatting, so it doesn't
interfere with the loading. This avoid opening a dialog when scrolling
the table over a still not fetched area with conditional formats.
And it is done independently of the threading model supported by SQLite.
This adds a label to the status bar which becomes visible whenever the
database is busy. It also specifies the currently running action. Also
add a button which allows you to cancel this action at the next
opportunity.
Note that this doesn't work for the Execute SQL tab yet which requires
some more rewriting of the code there.
After setting a filter, the user can select from the context menu in the
filter line a new option "Use for Conditional Format", that assigns
automatically a colour to the background of cells fulfilling that
condition.
The formatting is preserved after the user has removed the filter. Several
conditional formats can be successively added to a column using different
filters.
The conditional formats of a column can be cleared when the filter is empty
selecting "Clear All Conditional Formats" from the filter line context
menu.
The conditional formats are saved and loaded in project files as other
browse table settings.
A new class Palette has been added for reusing the automatic colour
assignment of the Plot Dock. It takes into account the theme kind of the
application (dark, light) for the colour selection.
A new class CondFormat for using the conditional formatting settings from
several classes. The conversion of a filter string from our format to an
SQL condition has been moved here for reuse in filters and conditional
formatting.
Whether the conditional format applies is resolved by SQLite, so filters
and conditional formats give the same exact results.
Code for getting a pragma value has been reused for getting the condition
result, and consequently renamed to selectSingleCell.
Possible future improvement:
- New dialog for editing the conditional formatting (at least colour and
application order of conditions, but maybe too: adding new conditions and
editing the condition itself).
SQLite 3.25.0 introduced an extended ALTER TABLE command which now
allows renaming an existing field. Before this we were emulating this
functionality in our code. There are however three reasons to switch to
the new feature from SQLite, even though it doesn't safe us any code:
1) It is faster because it does less steps
2) It is less error prone for the same reason
3) It is better at also renaming the field in triggers and views
This is somewhat improving the situation in issue #1444 but not
addressing the main problem described there.
The message when there are problems and the user has made changes and
accepted them must be different to the case when the user has cancelled
and any possible changes have been already reverted by the dialog.
In the first case the fault may be ours and we actually revert changes.
In the second case, we know for sure the problems were there before
editing the table. We warn user about it, since we already know.
The last error from the DB is inappropriate for both cases because it
hasn't been updated by the foreign-key check.
Also fixed the typo in a function identifier.
Room for improvement: maybe the check should be done before opening the
dialog, so we don't let the user edit the table until the issues have
been solved.
See issue #1547
This commit refactors vast parts of the sqlitetypes.h interface. Its
main goals are: less code, easier code, a more modern interface, reduced
likelihood for strange errors and more flexibility for future
extensions.
The main reason why the sqlitetypes.h functions were working so well in
DB4S was not because they were that stable but because they were
extremely interlinked with the rest of the code. This is fine because we
do not plan to ship them as a separate library. But it makes it hard to
find the obvious spot to fix an issue or to put a new function. It can
always be done in the sqlitetypes function or in the rest of the DB4S
code because it is just not clear what the interface between the two
should look like. This is supposed to be improved by this commit. One
main thing here is to make ownership of objects a bit clearer.
In theory the new code should be faster too but that difference will be
neglectable from a user POV.
This commit also fixes a hidden bug which caused all table constraints
to be removed in the Edit Table dialog when a single field was removed
from the table.
This is all still WIP and more work is needed to be done here.
In the Preferences dialog we allow the user to configure a list of
SQLite extensions which should be loaded whenever a new database file is
create or an existing database file is loaded. This commit changes the
order of actions after creating or opening a file so that the extensions
are loaded significantly earlier. This way they are already loaded for
most of the time during the process.
This fixes some subtle bugs. One of the more prominent ones was
triggered when there is a view which uses some function which is
defined in such an extension. After loading the file, parsing would fail
and you couldn't see the fields of the view. Browsing it would work but
trying to edit the display format crashed the application.
When exporting a database to an SQL file we need to query the row count
of each table to calculate the progress later in the actual export. This
commit replaces the SqliteTableModel class which was used for this
purpose by a simple query function. This might be slightly faster here
but more importantly avoids a progress window being shown for each table
before finally seeing the one for the actual export.
See issue #1476.
* Rename confusing variables
* Fix some project warnings
* Fix code style
* Add constant for the default page size
* Move KeyFormats enum to CipherSettings
* Fix code style
* Fix memory leak
* Stop relying on CipherDialog for encryption settings management
* Fix code style
* Add .env format for QSettings
* Add automatic crypted databases open via dotenvs
This adds support for `.env` files next to the crypted databases that
are to be opened that contains the needed cipher settings.
The only required one is the plain-text password as a value for the key
with the name of the database like this:
myCryptedDatabase.sqlite = MyPassword
This way, databases with a different extension are supported too:
myCryptedDatabase.db = MyPassword
You can also specify a custom page size adding a different line
(anywhere in the file) like this:
myCryptedDatabase.db_pageSize = 2048
If not specified, `1024` is used.
You can also specify the format of the specified key using the
associated integer id:
anotherCryptedDatabase.sqlite = 0xCAFEBABE
anotherCryptedDatabase.sqlite_keyFormat = 1
where `1` means a Raw key. If not specified, `0` is used, which means a
simple text Passphrase.
Dotenv files (`.env`) are already used on other platforms and by
different tools to manage environment variables, and it's recommended
to be ignored from version control systems, so they won't leak.
* Add new files to CMakeLists
* Move DotenvFormat include to the implementation
* Fix build error
* Remove superfluous method
(related to ac51c23)
* Remove superfluous checks
* Fix memory leaks
(introduced by 94bbb46)
* Fix code style
* Make dotenv related variable and comment clearer
* Remove duplicated code
* Remove unused forward declaration
(introduced by e5a0293)
Make strings translatable, remove some more debug code, fix tests,
reduce size of patch slightly, remove weird tooltip, don't crash when
closing database, simplify code, fix filters, don't link agains pthread
on Windows.
If a view has been unlocked for editing by specifiying a pseudo primary
key, with this commit you can now delete records from the view if an
appropriate trigger exists.
See issue #141.
When exporting a database to an SQL file we used to export it like this:
- Table 1 schema
- Table 1 data
- Table 2 schema
- Table 2 data
With this commit that is changed like this:
- Table 1 schema
- Table 2 schema
- Table 1 data
- Table 2 data
This makes the resulting SQL file more robust for import because it
avoids any foreign key errors as long as foreign keys are deferred.
Fix our default collation which we install when an unknown collation is
requested to (more or less) support UTF16 strings as well. Before this
UTF16 strings wouldn't compare correctly because they might contain null
bytes which would lead to only the first (few) byte(s) getting compared.
See issue #1172.
The no collate function warning is triggered in a thread which is used
for loading data. However, the warning is a message box and GUI elements
can only be drawn in the main thread. So the old code would crash. This
is fixed here by jumping to the main thread for showing the message box.
