DBF Append Speed in X# vs VO - Huge Difference

This forum is meant for questions and discussions about the X# language and tools
Jamal
Posts: 322
Joined: Mon Jul 03, 2017 7:02 pm

DBF Append Speed in X# vs VO - Huge Difference

Post by Jamal »

Robert,

Just wanted to let you know that the DBF speed issue persists in X# 2.8a.

Jamal
robert wrote:Jamal,
We'll look at this asap. Not for the 2.6 build that is coming in a few days, but in the build after that.
I know that VO used very aggressive caching for files opened exclusively, like in your example.
We'll try to build something similar in X#.

Robert
ic2
Posts: 1861
Joined: Sun Feb 28, 2016 11:30 pm
Location: Holland

DBF Append Speed in X# vs VO - Huge Difference

Post by ic2 »

Hello Jamal,

That is quite frustrating. We have a few programs, 1 exe and the rest in DLL's, using DBF's, but these are not data intensive.

I wonder if the speed issue also comes up when using ADS? Do you happen to have a test program that compares VO & X#? Then I can run it using X# native ADS access and with the Vulcan RDD as well and give you the results, if you can publish the code.

We can not afford to move to X# if there's a considerable append speed issue

Dick
Jamal
Posts: 322
Joined: Mon Jul 03, 2017 7:02 pm

DBF Append Speed in X# vs VO - Huge Difference

Post by Jamal »

Dick,

Just use the code I posted above (direct link): https://www.xsharp.eu/forum/public-prod ... ence#15885

ic2 wrote:Hello Jamal,

That is quite frustrating. We have a few programs, 1 exe and the rest in DLL's, using DBF's, but these are not data intensive.

I wonder if the speed issue also comes up when using ADS? Do you happen to have a test program that compares VO & X#? Then I can run it using X# native ADS access and with the Vulcan RDD as well and give you the results, if you can publish the code.

We can not afford to move to X# if there's a considerable append speed issue

Dick
User avatar
Chris
Posts: 4953
Joined: Thu Oct 08, 2015 7:48 am
Location: Greece

DBF Append Speed in X# vs VO - Huge Difference

Post by Chris »

Hi Jamal,

Indeed, there still seems to be a problem, in VO the time needed/no of records is always a linear function, while in X# speed decreases exponentially after around 100,000 records.

For smaller number of total records, X# speed is comparable to that of VO, although of course it can never become quite as fast, due to the overhead of .Net.
Chris Pyrgas

XSharp Development Team
chris(at)xsharp.eu
Jamal
Posts: 322
Joined: Mon Jul 03, 2017 7:02 pm

DBF Append Speed in X# vs VO - Huge Difference

Post by Jamal »

Hi Chris,

That's what my testing also showed.

This becomes a real issue when doing database bulk operations such importing data or updating existing rows.

I wonder if Vulcan.NET has such an issue.

Jamal
User avatar
Chris
Posts: 4953
Joined: Thu Oct 08, 2015 7:48 am
Location: Greece

DBF Append Speed in X# vs VO - Huge Difference

Post by Chris »

Hi Jamal,

No, vulcan seems it does not suffer from this exponential slow down. It is also quite faster in general, but then again the CDX RDD in vulcan is written in managed c++, which makes a big difference. I am sure it will be further improved also in X#, it's just the usual problem of having only 24 hours in a day... Still, RDD speed in X# has improved a lot in the last builds, in different scenarios than that of your sample.

Btw, to answer you question Dick, dbf speed with ADS should not have a difference between X# and VO, since all the work is done in the server by ADS itself.
Chris Pyrgas

XSharp Development Team
chris(at)xsharp.eu
Jamal
Posts: 322
Joined: Mon Jul 03, 2017 7:02 pm

DBF Append Speed in X# vs VO - Huge Difference

Post by Jamal »

Hi Chris,

Despite the fact I reported this issue last September, I can appreciate the constraint of time, however, I hope the dev team improves this situation to almost a few milliseconds (or seconds) of VO speed.

Jamal
User avatar
Chris
Posts: 4953
Joined: Thu Oct 08, 2015 7:48 am
Location: Greece

DBF Append Speed in X# vs VO - Huge Difference

Post by Chris »

Hi Jamal,

I am afraid this will be totally impossible. Even vulcan, in which the RDD source code is an almost 1-1 copy of the VO code, compiled in managed c++ with max level of optimizations, does not come anywhere close to that, due to the .Net overhead.

.Net in so many things is far superior to Win32, but when it comes to very low level code, it has a clear performance disadvantage by definition, so I am afraid we have to live with the fact that dbf usage will be slower. Having said that, since you (and others) have reported those speed issues, performance has already actually improved a lot. But I agree there' still enough room of improvement, especially in this case where something is causing this exponential increase in needed time to perform the operations.
Chris Pyrgas

XSharp Development Team
chris(at)xsharp.eu
Jamal
Posts: 322
Joined: Mon Jul 03, 2017 7:02 pm

Re: DBF Append Speed in X# vs VO - Huge Difference

Post by Jamal »

Hi All,

X# DBFCDX RDD Speed still exists in X# 2.18 public.

I really do appreciate the hard work that you have put in X#, however, for me this is a show stopper since it is a huge step backward and thus cannot adopt X# in real world applications and it keeps me on the fence.
I heard your reasons as was discussed in previous posts and forum discussions that VO is superior because it uses managed C++ code and .NET has an overhead which causes the slow down. With every X# update, I get excited and I run the same speed tests, unfortunately I see no change in performance.

In the past few months, I read somewhere on the forum that you are planning a DBFCDX Server option which will compete with Advantage Server? Is this still being considered and do you think the speed issue will be greatly minimized?

Best Regards :)
Jamal
User avatar
Horst
Posts: 337
Joined: Tue Oct 13, 2015 3:27 pm

Re: DBF Append Speed in X# vs VO - Huge Difference

Post by Horst »

Hello

I let it run also.

VO 15s vs X# 78s

I was reading the whole post, and i am cunfused about Kees post from 2020:
I think all the changes and improvements in 2.6 are less important than a reliable DBF/CDX system. I can't use X# because indexes get corrupted even when using only basic functionality. Yet more and more new features are added while this is just not working properly.

Is this fixed ? Because i am using also dbf/cdx and sometimes i have wired behavior.
Last time , i was deleting a record with dbAdministrator and then the Index was corrupted. i thought its the tool, but now im not sure :-(
Horst
Post Reply