Commit Graph

6 Commits

Author SHA1 Message Date
Félix Baylac-Jacqué 6787e782da Add an index on the full_insee field
This DB is mostly used to retrieve all the street names for a given
city. Adding an index for the full_insee field greatly improves the
query time.

Without the index:

time $(echo 'select * from fantoir where full_insee="65054";' | sqlite3 fantoir.sqlite)

real	0m2,230s
user	0m1,896s
sys	0m0,337s

With the index:

time $(echo 'select * from fantoir where full_insee=65054;' | sqlite3 fantoir.sqlite)

real	0m0,148s
user	0m0,126s
sys	0m0,024s
2022-02-17 22:18:23 +01:00
Félix Baylac-Jacqué 3bd8aa86bc Adding street transcodification
The fantoir label alone isn't enough to fully qualify a street name.
We need to take into account the street type and reflect it to the
final street name.
2022-02-17 22:18:13 +01:00
Félix Baylac-Jacqué d4441e93fa
Please read me 2021-11-17 21:44:20 +01:00
Félix Baylac-Jacqué 152d9d81b5
CSV intermediate file approach.
time ./scripts/gen-fantoir.sh FANTOIR0721
real	0m13,319s
user	0m9,804s
sys	0m3,458s
2021-11-17 21:32:02 +01:00
Félix Baylac-Jacqué 33c3c08d52
Naive SQLite implementation
Does not even terminate after a full night worth of run (~12h).

I probably should implement some kind of progress bar to estimate the
total run duration.
2021-11-17 20:45:16 +01:00
Félix Baylac-Jacqué e39f94bd25
Reference read
Perf:

$ time target/release/fast-fantoir FANTOIR0721

real	0m0,608s
user	0m0,526s
sys	0m0,081s

$ time target/debug/fast-fantoir FANTOIR0721

real	0m11,664s
user	0m11,549s
sys	0m0,113s

I highly suspect the release build to optimize out the useless file
read, partially explaining the *MASSIVE* perf difference between the
release build and the debug one.

This gives up a baseline speed order though. We should probably be
able to stay below 1 minute to generate the DB.
2021-11-03 21:27:07 +01:00