When parsing a CSV file we used to check the column count for each row
and track the highest number of columns that we found. This information
then could be used to create an INSERT statement large enough for all
the data.
This column number tracking code is removed by this commit. Instead it
analyses the first 20 rows only. It does that while generating the field
list.
Performance-wise this should take a (very) little longer but makes it
easier to improve the performance in other ways later which should more
than compensate this commit.
Feature-wise this should fix some (technically invalid) corner-case CSV
files with fewer fields in the title row than in the other rows. It
should also break some other (technically invalid) corner-case CSV files
if they are imported into an existing table and have less columns than
the existing table in their first 20 rows but later on the exact same
number. Both cases, I think, don't matter too much.
Don't build a separate SQL statement per row to insert during CSV import
but use a single prepared statement which can be reused for each row.
This should speed up the CSV import noticeably.
This adds initial basic support for handling different database schemata
at once to the backend code. This is still far from working properly but
shouldn't break much either - mostly because it's not really used yet in
the user interface code.
When importing a CSV file into an existing table (i.e. a table where we
have a table schema), check the data type of a field before inserting
empty values. If it is an integer field, don't insert empty string like
we did before but 0 or NULL depending on the NOT NULL flag.
See issue #195.
Please note that this isn't perfect. The preview in the dialog doesn't
reflect these changes yet, it just show you the contents of the file as
is. It's a little tricky to change this and I somehow think it's better
the way it is now anyway. Also the import doesn't check for other
constraints like UNIQUE or CHECK which might cause trouble. But then
again it didn't do that before either.
When importing a CSV file and using the first row as the field names,
the row would be imported as the first data row again. It's now skipped
when the checkbox is set.
When importing a single CSV file the checkbox asking whether to import
into a single table or separate tables is hidden. However, the last set
values are loaded anyway when the dialog is opened. This means the
checkbox could be set, even though it's invisible. If it's set, however,
and we're importing a single CSV file this would mean that it's
impossible to manually set the table name to import into. This is fixed,
too.
Also this simplifies the code a bit and removed a large loop from the
import dialog code.
- Tweak input checker
- Preserve old file import as not to cause any unforeseen breaks
- Allow ignoring file name when importing multiple files to tables
- Mass toggle several files for import
This finally gets rid of the DBBrowserObject class entirely and moves
all its functionality to the newer classes in the sqlb namespace.
I'm still not entirely happy with this but at least things should be a
little more consistent now.
This changes the class structure in the sqlb namespace as well as the
DBBrowserObject class. The rest of the commit are changes that are
required by the modifications in sqlb and DBBrowserObject.
The idea behind this refactoring is this: we currently have the
DBBrowserObject class which holds some basic information about the
database object (name, type, SQL string, etc.). It also contains a
sqlb::Table and a sqlb::Index object. Those are used if the type of
the object is table or index and they contain a whole lot more
information on the object than the DBBrowserObject class, including the
name, the type, the SQL string, etc.
So we have a duplication here. There are two class structures for
storing the same information. This has historic reasons but other than
that there is no point in keeping it this way. With this commit I start
the work of consolidating the sqlb classes in order to get rid of the
DBBrowserObject class entirely.
This commit only starts this task, it doesn't finish it. This is why it
is a little messy here and there, but then again the old structure was a
little messy, too. We will need at least a very basic trigger and view
parser before finishing this is even possible. When this is done, I hope
the ode will be much easier to read and understand. But even in the
current state there already is some progress: we save a little bit of
memory, don't copy big objects all the time anymore, and replace a lot
of unnecessary string comparisons with integer comparisons.
When generating SQL statements properly escape all identifiers, even
those containing backticks which apparently are allowed inside
identifiers in SQLite.
See issue #387.
Add some basic initial support for SQLCipher. Note that this is more of
a POC than a final implementation.
This commit adds an option called 'sqlcipher' to the cmake and qmake
projects which - when enabled - replaces the default SQLite3 include and
library files by their SQLCipher counter-parts. Especially on MacOS X
there might be some more work required in finding the correct include
paths. The SQLCipher library supports unencrypted databases, too, so
even if the option is enabled the program behaves like before. You can
see the difference, though, in the About Dialog where the SQLite version
string will say 'SQLCipher version xy'.
When the sqlcipher option is enabled and you try to open a file which is
neither a project file nor a normal SQLite3 database it is assumed now
that the file is an encypted database. There is no way to tell between
an invalid file and an encypted file, so in both cases a password dialog
pops up. When the correct password and page size are entered the file is
opened and can be edited like any other database before.
Creating encrypted databases isn't supported yet. So for testing you
need to fall back to the sqlcipher command line tool.
See issue #12.
No need to update the progress window after each row. It's only really
visible for large files anyway where a single row doesn't matter that
much in terms of progress.
Also fix a compiler warning.
Keep all the table and field information in a sqlb::Table object (which
itself is stored in DBBrowserObject) rather than storing field
information in DBBrowserObject and sqlb::Table objects at the same time.
Make it possible to use other quote and separator characters than those
predefined in the respective combo boxes when importing a CSV file.
See issue #23.
Spaces in table, index and field names are actually allowed by SQLite,
so no need to check for them.
However, "`" characters cause problems when appearing in SQL statements
so disallow them instead.
Allow setting the default value, the check values and under some
circumstances the not null flag when editing existing tables.
This required some changes to DBBrowserDB::renameColumn() which is now
using some more features of sqlitetypes.cpp but could still be improved.
Do just the same in the ImportCsvDialog as before in the
EditTableDialog, i.e. using a unique savepoint name which is not
released when the import was successfull.
Rewrite the dialog to import CSV files using a Qt Creator form file.
Simplify the CSV import a little bit.
A bit unrelated, but needed for this anyway:
Create the restorepoint right after opening a database file instead of
just doing so after the first change.
