Import ASCII-file 150.000 lines (50 MB)

Luc_VanDyck

I need to import a ASCII-file with 150.000 lines (size 50 MB) into Navision Financials 2.60.

The recordsize is > 250 chars.

What is the fastest way to do the import: using dataports or writing a codeunit to read the file?

mfabian

A dataport because in codeunits you can read files with reclen>250 only binary. And that's quite slow.

Marcus

---

Marcus Fabian
m.fabian@thenet.ch
+41 79 439 78 72

Luc_VanDyck

I have written a dataport for importing this file. The dataportfield-list contains 4 txt-variables (250 chars.).

It works ... but it is VERY slow. I have already splitted the file into 4 files (each ca. 14 MB), but there is no much difference in speed.

Is there anything else I can do to improve performance?

mfabian

If huge amount of data is involved, Navision slows down considerably during processing. Typical behaviour: in the beginning it's very fast but after a while it slows down more and more.

What I could recommend at this point are two things:

1) Increase the cache as high as possible. This allowes a little bit more records to import before the cache is filled and the system will slow down.

2) Providing the logic of your application allows, you could make regulary COMMIT's - let's say every 10'000 records to free the cache.

Probably a third (funny) approach: Try to import the same db in an access database. After that you won't say anymore that Navision is slow :-)

Marcus

---

Marcus Fabian
m.fabian@thenet.ch
+41 79 439 78 72

db

Try use CFront.
If averige hdd access ~15ms max transfer 40 rec/s. (measured on table 32).
Speed of writing only decrasing then free space on fdb is missed.
You need free space on fdb = 50 MB + 10% of all fdb size.

Luc_VanDyck

I have imported this 50MB file in 1 (one) hour!

The trick is to split the file into 46 files (4000 lines each).
Then you write a dataport that starts importing from file 1, and then moves on with the next file until all the files are processed.