I'm looking for pro and cons in webservices compared to NAS

Sunset

I've been asked for some documentation about pros and cons of NAS and webservices that are relevant for both technical level people, and executive level. I've tried looking around partnersource, but couldn't find anything useful (then again I've never succesfully found anything there).
Does anybody here have something that I might take a peek at?

kriki

Here are some:

-NAS: 1 task at a time per NAS
+WS: multiple tasks in parallel possible if they don't lock each other (e.g. multiple tasks that post invoices cannot work in parallel because of locking of the entry-tables).

+NAS: active: starts doing something when he decides (e.g. waiting for a message through message-queues or triggered by a timer).
-WS: passive: works only when called

-NAS: a NAS works only in 1 company defined at installation (or startup of NAS).
+WS: can work in any company

+NAS: works with both native as SQL DB
-WS: works only with SQL DB

-NAS: cannot be used by RTC
+WS: can be used by RTC (MANDATORY!)

+NAS: versions 3.01 => NAV2009SP1 (it is the latest version at date)
-WS: only versions NAV2009+

Sunset

Thank you kriki, anybody got more?

ara3n

You can use NAS and Webservices combined. An example would be.

http://mibuso.com/blogs/ara3n/2009/01/2 ... companies/





WS + requires much less programming in NAV side.

David_Singleton

Hey Alain that is a really good technical comparison. :thumbsup:

Let me add from the "why you would and why you wouldn't" view point.

For me I would use Web Services were I need a lot of flexibility and performance is not a major issue. And more importantly I would use it where I need to work with developers that have no clue about Navision.

NAS I would use for industrial stuff, where I have a specific requirement to transfer data or post specific document or journals and I need to control what posts when and where for performance reasons. But for sure I would want Navision people on both sides of the interface project.

Sunset

=D> =D> I'm loving it =D> =D>

Keep it coming :thumbsup:

ara3n

David are you saying that webservice performance is worse than NAS?


Webservice is multi-threaded and thus can accept many connection and you can scale it.


With NAS, you have to purchase new nas license for every new thread. $$$

Sunset

ara3n wrote:

David are you saying that webservice performance is worse than NAS?...

I would hazard to guess that what David is referring is that where you can afford a timeout/retry while handling "fancy stuff" you are fine with webservices, whereas if you need the low-tech stability you might be better of with NAS.
For instance if you have a webshop with multible querries to stock and prices, you can afford the occasional timeout when calculating sales price, stock, delivery time etc.
But when handling data-synchronisation between systems you might be better served with NAS.

David_Singleton

Sorry I knew as soon as I posted that I was not clear. :oops:

No of course there is little absolute performance difference in performing the same function. But what I have seen so far, the people writing web services are often not Navision people, so they do things they way "they" would do it not the way we would do it. As Sunset points out people are going to use Web services for a lot of add hock stuff in a very asynchronous manner, so you are going to see lots of locking etc. If its NAS then you would be doing things sequentially and thus blocking is not as big an issue.

But assuming one NAS running one task vs one Web Service running the same one task, there would not be a great difference I imagine.

ara3n

Sunset wrote:

ara3n wrote:

David are you saying that webservice performance is worse than NAS?...

I would hazard to guess that what David is referring is that where you can afford a timeout/retry while handling "fancy stuff" you are fine with webservices, whereas if you need the low-tech stability you might be better of with NAS.
For instance if you have a webshop with multible querries to stock and prices, you can afford the occasional timeout when calculating sales price, stock, delivery time etc.
But when handling data-synchronisation between systems you might be better served with NAS.

With NAS, you still have to use some other solution for data-synchronization. Be it msmq or file based or what not.
You can do the same thing with webservices. The issue is simply that webservice is passive, and so is NAS.

For NAS the solution has been built using Timer, but you can do the same thing with SQL Jobs, or another windows service that behaves as a timer and calls a webservice or waits for some events.

Granted new solutions, majority of the solutions involves developers that work outside of NAV, but it doesn't mean you can develop the nav way using webservices.

Romul

kriki wrote:

Here are some:
-WS: passive: works only when called

It's not a lack absolutely because you can write windows service calling your web services. And it will work permanently or periodically. As you wish...
Costing Adjustment is example of it:
1. Windows Service with itself application logic (with timer or with schedule or something else...........);
2. Web Service with costing adjustment functionality;
3. Windows Service calls to Web Service. Windows Service is stopped when costing adjustment processing finishes.

kriki wrote:

-WS: works only with SQL DB

Wow... The biggest lack.
Read it please:
http://blogs.msdn.com/nav/archive/2010/ ... abase.aspx

kriki wrote:

-NAS: cannot be used by RTC

It's NOT NECESSARY to use NAS in 2009 NAV! You can write countless of windows services which call your web services! NAS is atavism. It SHOULD NOT be puchased by clever client if he wants to save his money and nerves.

kriki wrote:

+NAS: versions 3.01 => NAV2009SP1 (it is the latest version at date)

I'll repeat it... There are no reasons to use NAS in NAV 2009 or NAV 2009 SP1. None.
You can use Windows Services and Web Services even in ADCS (Automation Data Capture System) for NAV.

kriki wrote:

-WS: only versions NAV2009+

Not a lack. For old version of NAV you have NAS :thumbsdown:
For NAV 2009 and NAV 2009 SP1 you have Web Services and Windows Services. :thumbsup:

David_Singleton

ara3n wrote:

Granted new solutions, majority of the solutions involves developers that work outside of NAV, but it doesn't mean you can develop the nav way using webservices.

Which is basically what I said. It will make people lazy.

Once the performance is up on the service tier, things will be different of course, but in the current version there is still a need for NAS in some applications.

rdebath

ara3n wrote:

WS + requires much less programming in NAV side.

Use the job queue, for the NAS side. You get a scheduler or batch processing for a programming cost the same or smaller than a webservice codeunit/page.

Romul wrote:

It SHOULD NOT be puchased by clever client if he wants to save his money and nerves.

Oddly enough I think I agree with this; thanks for making it clear. (OTOH I'm currently convinced that most of our customers would HATE the RTC six months down the line because we'll have to keep saying "Microsoft don't allow us to do that in the RTC" we'll have to do it in C# instead so it's gonna cost three (ten?) times as much.)

This is my list of things the NAS can do with COM objects and how to do them with the WS.
MS-SQL's scheduler or the windows task scheduler can be made to call a webservice for timed operations. A proxy webservice is probably better than a MSMQ for external systems making calls to Navision (it has to use a proxy because the NAV WS uses 'Microsoft domain authentication' not any standard method). The only other problem is 'fire and forget' style MSMQ communications which can be sent from RTC code without change and polled for from a WS for inbound.

My only worry is running long jobs, Romul, how would you suggest I reliably run a batch job (eg adjust cost item entries) that can reasonably take hours to run? As David said, webservice calls time out...

Romul

rdebath wrote:

Romul wrote:

It SHOULD NOT be puchased by clever client if he wants to save his money and nerves.

Oddly enough I think I agree with this; thanks for making it clear. (OTOH I'm currently convinced that most of our customers would HATE the RTC six months down the line because we'll have to keep saying "Microsoft don't allow us to do that in the RTC" we'll have to do it in C# instead so it's gonna cost three (ten?) times as much.)

rdebath, I mean that client shouldn't buy NAS

rdebath wrote:

My only worry is running long jobs, Romul, how would you suggest I reliably run a batch job (eg adjust cost item entries) that can reasonably take hours to run? As David said, webservice calls time out...

Well... Really, I don't see reasons not to do it. I have my own C# Windows Service and Web Service for cost adjusting. It works fine. Despite of absense of NAV 2009 in Russia yet.

rdebath

Romul wrote:

Well... Really, I don't see reasons not to do it. I have my own C# Windows Service and Web Service for cost adjusting. It works fine. Despite of absense of NAV 2009 in Russia yet.

You ... have ... your ... own ... routine ... Oh boy! 8-[ Have you heard of transactions? [-o< Do your customers know?

Maybe one of freddyk's calling methods will wait forever. :-k

Romul

rdebath wrote:

Romul wrote:

Well... Really, I don't see reasons not to do it. I have my own C# Windows Service and Web Service for cost adjusting. It works fine. Despite of absense of NAV 2009 in Russia yet.

You ... have ... your ... own ... routine ... Oh boy! 8-[ Have you heard of transactions? [-o< Do your customers know?

Maybe one of freddyk's calling methods will wait forever. :-k

No comment...
Yes, our customers know it. Do you have a customer with > 1Tb database? Oh boy... I think you haven't...

kriki

rdebath wrote:

My only worry is running long jobs, Romul, how would you suggest I reliably run a batch job (eg adjust cost item entries) that can reasonably take hours to run? As David said, webservice calls time out...

Actually, I am using ara3n's solution with job queues that call webservices. I did change something so the jobqueue (=NAS) does NOT wait for the WS to finish.
And we are running adjust cost item entries + posting them to G/L in 1 WS call and (after fixing the disabled picture-field in table 27 in NAV2009-bug, see http://mibuso.com/blogs/kriki/2010/02/17/am-i-a-detective/) all works just fine!

Sunset

Not sure if Romul just proved David's point about some people programming Navision style, and others don't.
Personally I would be extremely reluctant of putting something as crucial as cost adjustment outside NAV, but if you have terabytes of item transaction i could perhaps be pressed to see the reason for it.
Then again, after 18 years in NAV I might just be seing ghosts after having experienced way to many near-death solutions from inexperienced NAV consultants

ara3n

Could you provide some info on 1TB db on hardware, and type of transaction they are posting, costing method, no of record for ILE, GL, Item,customer?

I'm sure MS would want to use it to improve performance of NAV..

rdebath

ara3n wrote:

I'm sure MS would want to use it to improve performance of NAV..

And not only MS, I'd love to have real data to push through the performance test tool for a DB of that size.

Romul

ara3n wrote:

Could you provide some info on 1TB db on hardware, and type of transaction they are posting, costing method, no of record for ILE, GL, Item,customer?

I'm sure MS would want to use it to improve performance of NAV..

It's impossible to discuss about technical features of this DB because of contract obligation...

ara3n

well don't talk about this DB, instead talk about a new client that is going to have 1TB and what hardware it going to have.
What kind of tunning/ mods you have to do to get to run their business on NAV.

Marije_Brummel

I do not think that this will lead to a usefull list. I have worked on noumerous large systems and amongst them >1TB systems and none of them have the same tuning.

Some have tons of value entries, others have customer ledger entries, others have whatever other data.

Tuning also depens enourmously on the finsql.exe and SQL version. Older runtimes need different tuning/indexes than newer.

The hardware also depens on the load of the system. These can be RAID 5 systems with mirrored HP EVA's with large caching/commit systems like falconstor to help with the checksum calculations.

jimclarkus

Romul wrote:

rdebath wrote:

Romul wrote:

It SHOULD NOT be puchased by clever client if he wants to save his money and nerves.

Oddly enough I think I agree with this; thanks for making it clear. (OTOH I'm currently convinced that most of our customers would HATE the RTC six months down the line because we'll have to keep saying "Microsoft don't allow us to do that in the RTC" we'll have to do it in C# instead so it's gonna cost three (ten?) times as much.)

rdebath, I mean that client shouldn't buy NAS

rdebath wrote:

My only worry is running long jobs, Romul, how would you suggest I reliably run a batch job (eg adjust cost item entries) that can reasonably take hours to run? As David said, webservice calls time out...

Well... Really, I don't see reasons not to do it. I have my own C# Windows Service and Web Service for cost adjusting. It works fine. Despite of absense of NAV 2009 in Russia yet.

I have my own C# Windows Service and Web Service for cost adjusting
Is it really possible?

rdebath

jimclarkus wrote:

I have my own C# Windows Service and Web Service for cost adjusting
Is it really possible?

Yes of course it is, any of the business code in Navision can be replaced or rewritten. But it's probably a very bad idea because the code in the standard "Adjust Cost Item Entries" routine has been around for quite a while, it's complex and still has bugs being fixed regularly. When you translate or rewrite this code you're very likely to add new bugs and it's very unlikely that you're going to have the time, money or experience to find them before 'go live'.

If you have a definite goal in mind it can be worthwhile to rewrite key routines; for example rewriting things in Transact SQL can to wonders for performance. But IMO Acie is unlikely to gain anything from a C# transformation and the cost from it's complexity normally precludes translation into T-SQL. I would say a good route would be to stay in C/SIDE and reduce the impact of Acie so that it can be run any time by reducing the size of the transactions that the routine uses.

A direct example; for the RTC all the code in Navision is translated into C# code, (rather nasty code but that's beside the point) the speed of tight loops in the translated C/SIDE code running within the service tier is better than ten times that of the interpreted C/SIDE code in the classic client. But it makes almost no difference to the actual measured times of user tasks; in fact unless you're very careful with the setup users are likely to complain that the RTC feels a lot slower.

So yes it's possible, but no I think it's a very bad idea.

generic

rdebath wrote:

So yes it's possible, but no I think it's a very bad idea.

So what would be your solution if it takes adjust cost 10 hours to run? Tell the client that they have bought the wrong ERP and should to Oracle or SAP?

The weakness of NAV is performance/scalability , and it's a natural side effect of simplicity.

Sadly simplicity is was lost in the area of SN/Lot tracking. MRP.
Costing itself is a complex matter, but NAV team has done a great job in simplifying and clarifying it in v 5.0 and up.
Kudos to them. =D> =D> =D>

rdebath

I would be moving it closer to the database, initially physically by running it in an application server on the same machine as the DB server. This might not gain you much though. After that, as I said, I'd be either moving it actually into the DB server by converting it to T-SQL (now I think of it C# can be used in stored procedures too) or I'd be leaving it as C/SIDE and reducing the transaction size and work to do so it can be run anytime without killing the system. Possibly both so I don't have to put it all into stored procedures.

What I'd not be doing is moving the code further from the database by introducing an extra layer of indirection in the form of the webservices.

And yes, Microsoft have been doing a lot of work on this. So somewhere in there would be an upgrade to the most recent version of ACIE, just that object not the entire system, and probably a strong suggestion for a technical upgrade if they're a long way behind the bleeding edge.

But of course, the first thing is to monitor the update and actually see what's really making it take so long. If it's something silly like the the backups running at the same time there's nothing to fix... (I can hope can't I? )

Edit: Your second point.
The reason NAV doesn't scale well is mainly the "Giant Lock"

GLEntry.LOCKTABLE;
IF GLEntry.FINDLAST THEN;

One of these is taken out at the very start of every posting, this kills multi-user. I suppose this does simplify some things, you can basically ignore deadlocks, but the cost is so high I'd call it an oversimplification, because deadlocks can be avoided by simple rules. At least that would be completely true if "UpdateNoLocks" did what it says on the can, as it is you'd probably have to do some serious buffering of your updates too, which isn't so simple.

The S/N tracking issues may be part of the same problem, but I do note that if you are using S/N tracking every single S/N puts several large records into the database, this doesn't seem like a good idea. :-k

kriki

The LOCKTABLE+FINDLAST is an inheritance of the native DB that only knows table-locks.
When MS does stops using native DB and completely goes to SQL, they probably (hopefully) will fix it.

davmac1

The problem with the ledger table is posting several entries with consective numbers.
This would need a redesign to eliminate table locking.
One possible solution would be to use the transaction no. field instead to link common GL entries (plus CLE, VLE, and ILE).
Then the primary key could just be a SQL generated sequence number.

Sunset

Although we seem to be straying a bit of course from the original question; the discussion is at least an interesting one

There is no need in NAV to have the posting in consecutive numbers any longer. The old consistent check had some reasons for this, but as the new consistent only makes sure that debit and credit equals 0 for the posting routine you are currently in, the entry number is "reduced" to being a key. And as such there is no technical reason (at least to my knowledge) why a posting should have consecutive numbers.
For traceability it is a very nice feature, but as a technical reason it doesn't hold.

rdebath

Sunset wrote:

And as such there is no technical reason (at least to my knowledge) why a posting should have consecutive numbers.
For traceability it is a very nice feature, but as a technical reason it doesn't hold.

I have successfully altered the G/L posting to do this, I used the G/L register number as the link. You have to used a distinct table to collect the entry numbers as they must match between the various ledgers but it works fine for small journals. For larger journals Navision stops doing it's delayed insert thing and you start getting locking on the VSIFT indexes.

I've found that saving the postings into temp tables then allocating the set of entry numbers at the end of the posting gives better results and the entry numbers are still sequential. Posting about 120 sales invoices with 2-5 G/L lines plus the same number of matched customer payments and misc G/L postings every minute. This was on an ordinary (virtual, vmware) server with normal SAS drives, no special caching. Extra manual SI postings took a few seconds screen time each, as if on an unloaded system. (I haven't done the Item ledger, this customer doesn't need it but the theory is exactly the same)

My main bugbear is that UpdateNoLocks is a lie. If you touch any record in a table from then on everything you just look at in that table gets locks; plus records even close to them. This is a real pig with dimensions.

kriki wrote:

The LOCKTABLE+FINDLAST is an inheritance of the native DB that only knows table-locks.
When MS does stops using native DB and completely goes to SQL, they probably (hopefully) will fix it.

Not exactly an inheritance, but it is a very easy way to work around the deadlocks that record level locking showed up. However, there is a problem; my guesstimate would be that Navision will outperform Axapta with this fix ...