Migration needs 8 hours - any ideas?

baldaum

Hi,

we have done a migration test from 4.03 Native to 5.01 SQL. The database is about 50GB.
The migration is working fine, but the .fob field check needs about 8 hours. The problem is that the whole migration process needs about 15 hours (export, import, fob check, export, import, key generation, etc.)

Does anybody have an idea how to speed up the migration process? We have already tried to run NAV on a dual quadcore server, but that does not help. =D>

regards
Markus

ara3n

you only need to run fob check once and fix the dates. Once you've fixed them you don't need to run it again. Just tell the users to be careful on entering dates till you do the final migration.

The other option is to write a custom object that will only run for newly created transactions.

baldaum

Hi,

but if one user does not enter a correct data I have start completely from the scratch. This would mean that I have double work. I want to do the fob check just before migration.

Any other ideas?

regards
Markus

ara3n

The other option is to write a custom object that will only run for newly created transactions.

baldaum

Hi,

thank you very much for your help. Regarding your idea I have some questions:

1. Can I run the migrate.fob, which is shipped with the version 5.01, also on our live production database, which is native 4.03?
2. If I start the migrate.fob, which checks all fields, is it possible that our users can work in Navision. This would save 8 hours and I could do the field check some days before.
3. Who can write a custom migrate.fob? How do I check only the data which was entered after a special date?

Many thanks for your help!

Markus

DenSter

There is no shortcut, you have to run migrate.fob before the go-live upgrade. If there are transactions between the time that you run it and the time that production is migrated, there's always a risk that it blows up.

baldaum

Hi Daniel,

thx for your help.

This means that the migration will need 15 hous in total and there is no way to speed up?

regards
Markus

David_Singleton

baldaum wrote:

3. Who can write a custom migrate.fob? How do I check only the data which was entered after a special date?

I would say that a person capable of writing the code to do the upgrade is able to do this. So speak to who ever is doing the upgrade.

jre

Hello,

When migrating from NAV Ver 4.0 to NAV Ver 5.1 FOB migration and fieldcheck are done normally in the database converted to version 5.1. The users are working still on version 4.0 database until complete migration process is done. So doing the fieldcheck and correcting the date happens on version 5.1. Here it seems changing the script can no way help reducing the time taken.

Is it possible to do the Fieldcheck and date correction in NAV Ver 4.0? Any suggestions ??


Thanks,

JRE

bbrown

It's all about disk performance. Since a migration to SQL usually involves a new faster server, my approach is to leverage that in the upgrade process. In a recent upgrade, of a 50 GB database, utilizing the faster disk (and greater number) of the new server allowed for spreading the native database across more files. This greatly reduced time for both the data checks and the conversion itself.

baldaum

Hi,

so you would recommend restoring the native DB to a new server which faster discs and more DB-Files. The problem is that if we do an export of the native and then do a restore on the new server into more DB-Files this already takes 4 hours, which would not help.

We will win some time in the migration process, but we will loose time for the export and import to a new server.

regards
Markus

ara3n

15 hours is not an issue, are you a 24/7 company?
If you are not, then you have the whole weekend to do the migration.

You can also split the migration FOB into two CU and run them at the same time.

bbrown

In the example I cited the backup did take a few hours due to the older slower system. But the restore only took 20 minutes and all aspects of the upgrade were faster since they were on a faster system. That entire upgrade from starting backup thru restoring to SQL took under 20 hours.

baldaum

Hi,

the problem is that we are really a 24/7 company and it is a big problem to shutdown the system for 15 hours.
It is clear that we start on Friday evening with the migration, but we will not finish until Saturday at about 6 pm.

Any other ideas?

Many thanks
Markus

ara3n

As I mentioned you need to modify the migrate.fob so that it only looks at subset of records.

Or at least split the file in half and run it concurrently.

This way One process will look at half the files, while the other will look at the rest of the files.


You can actually split it more than 1 and let 4 or 10 clients to run at same time. If your server can handle it, it should run much faster.

matttrax

Yeah, there are plenty of ways to make it so that it doesn't check the same records again. Add a "checked" field to the table, store the "Last Entry Checked" in a custom table. Then you filter accordingly.

Obviously we don't know the specifics of your business, but if you can't be down there's always pen and paper and re-keying once it's up and running. Which if you're really a 24/7 company I'd imagine you'll do for the first few hours you're down anyway.

ara3n

Also if you are a 24/7 company, and down time is an issue, I suggest to contact your solution center and they will come up with a plan.

David_Singleton

In my experience with SQL conversions, I have never seen any date errors created by the users in the month or so before conversion. Generally the largest number of date errors come from the initial data conversion from the old system. So those can be found and fixed any time. Then the first 2-6 month there will be users from the old system that are still entering the dates incorrectly the same as they used to. Since Navision has lots of date based functionality, those users eventually learn. And then you have a few stragglers. So normally what I do is when the customer first says "should we move to SQL" I run the migrate process and get most of the dates sorted out. Then I run it again a month or so later to see if anyone is messing up. If they are then put in some simple code to stop them doing it.

But generally I don't have to run the migrate on conversion day.

ara3n

you can also modify in CU 1 function MakeDateText and check and make sure they enter a valid date.

That way you don't need to check anything and don't need to run the process twice.

David_Singleton

ara3n wrote:

you can also modify in CU 1 function MakeDateText and check and make sure they enter a valid date.

That way you don't need to check anything and don't need to run the process twice.

Thats what I meant by:

David Singleton wrote:

Then I run it again a month or so later to see if anyone is messing up. If they are then put in some simple code to stop them doing it.

jre

Hi,

The splitting of the fob file and running it concurrently looks good idea.

Regards,

JRE

ara3n

make sure you split so that large tables run on separate codeunits.

jre

Hi,

There are some problems I have seen.. First of all the CU 104015 can be updated separately for different instants of the navision and compiled and run. The licence doesnot allows to make Field Check in a different numbers other than 104015.

Secondly when the different instances are run concurrently, the Table locking is happening, so in effect only one instant is running others remains in still stand.

Any ideas to overcome these problems?

Regards,
JRE

David_Singleton

jre wrote:

Hi,

There are some problems I have seen.. First of all the CU 104015 can be updated separately for different instants of the navision and compiled and run. The licence doesnot allows to make Field Check in a different numbers other than 104015.

Secondly when the different instances are run concurrently, the Table locking is happening, so in effect only one instant is running others remains in still stand.

Any ideas to overcome these problems?

Regards,
JRE

David Singleton wrote:

In my experience with SQL conversions, I have never seen any date errors created by the users in the month or so before conversion. Generally the largest number of date errors come from the initial data conversion from the old system. So those can be found and fixed any time. Then the first 2-6 month there will be users from the old system that are still entering the dates incorrectly the same as they used to. Since Navision has lots of date based functionality, those users eventually learn. And then you have a few stragglers. So normally what I do is when the customer first says "should we move to SQL" I run the migrate process and get most of the dates sorted out. Then I run it again a month or so later to see if anyone is messing up. If they are then put in some simple code to stop them doing it.

But generally I don't have to run the migrate on conversion day.

jre

Hi David,

David you are telling about date fixing in the old system. Can you tell me how you do the date fixing on the old version?? on Native Database Navision 4.0?

Regards,
JRE

David_Singleton

jre wrote:

Hi David,

David you are telling about date fixing in the old system. Can you tell me how you do the date fixing on the old version?? on Native Database Navision 4.0?

Regards,
JRE

This entire thread is about exactly that. You MUST fix the dates on the old system BEFORE you can convert to SQL.

garak

jre wrote:

Hi David,

David you are telling about date fixing in the old system. Can you tell me how you do the date fixing on the old version?? on Native Database Navision 4.0?

Regards,
JRE

Read this also before. It could help you.

viewtopic.php?f=23&t=30050

And check the documentation on the Upgrade Toolkit.
Also if you searc hthe forum for "Migrate Native SQL" you will find a lot of topics about these theme. It's there ever the same question :-)

Regards

jre

Hello,

The topic you have given says about running the migrate.fob and then making the native database backup. It looks alright.

As per the migration object documentation which i have [SQLMigrationUpgradeToolkitManualChapter5.pdf ] , what I understood is that migration.fob from a converted database to Nav 5.0. Am I refering a wrong SQLMigrationUpgradeToolkitManual? May be I should referer a document for NAV Version 4.0 ?


Regards,
JRE