Is this possible on CDX ?

wriedmann · Post by **wriedmann** » Fri Jan 31, 2020 7:00 am

Hi Thomas,
unfortunately I don't know nothing about bitmap indices, but since the current X# RDD is built by Robert, I'm pretty sure he can add the neded things.
About Rushmore (and yes Karl, it was Uwe's article in the SDT I remember): AFAIK when a filter contained fields that were also present in the index, the filter engine evaluated first this part of the filter, and applied the filter than only on the already preselected records. For this purpose a bitmap was built in the memory.
When Microsoft acquired the company behind FoxPro, there were rumors that they had buyed the company only to have access to Rushmore so they could add this also to their Access database product.
Wolfgang

robert · Post by **robert** » Fri Jan 31, 2020 7:52 am

Guys,

We will look at optimized filters and similar things as well. We know what to do, it just takes time.

Robert

wriedmann · Post by **wriedmann** » Fri Jan 31, 2020 7:59 am

Hi Robert,

it just takes time

Agreed 100%!

Unfortunately your time is limited (as ours is too....)

Wolfgang

P.S. can we expect an interim X# release soon? I would need the new RDD with the new CopyDB functionality

robert · Post by **robert** » Fri Jan 31, 2020 6:27 pm

Wolfgang,
An intermediate build should be ready around Feb 15.

Robert

mainhatten · Post by **mainhatten** » Fri Jan 31, 2020 6:58 pm

Hi Wolfgang,
somewhat reordered...

wriedmann wrote:About Rushmore (and yes Karl, it was Uwe's article in the SDT I remember): AFAIK when a filter contained fields that were also present in the index, the filter engine evaluated first this part of the filter, and applied the filter than only on the already preselected records. For this purpose a bitmap was built in the memory

Close enough - the filter expression is checked - if it can be expressed via the precalculated order values in any currently connected index of possibly many .cdx/idx/ndx totally or at least in part, a bitmap is created, where bit position corresponds to recno and inclusion/exclusion is flagged via bit value for each such part. So if you have indeces on different (perhaps transposed "upper", "proper"..) columns, connected logically in the filter expression, for each such filter part a new bitmap is built. These bitmaps are added together, which is a fast operation, to get a bitmap of all recno() which need to be checked for filter conditions not adressed by any existing index - you could call it a prefiltered table scan.

unfortunately I don't know nothing about bitmap indices, but since the current X# RDD is built by Robert, I'm pretty sure he can add the neded things

.
Think of the vfp "binary" index as typical header with indexexpression and other info, but then instead of a tree with recno-key value tree just the materialized bitmap described above: bit position equates to recno(), flag value to key value. see
https://www.geeksforgeeks.org/bitmap-indexing-in-dbms/
https://stackoverflow.com/questions/345 ... es-helpful
as they describe it better than I could, also check out Wiki article linked in Stackoverflow. IMO hugest benefit (combining totally unrelated filter columns) is also mentioned there.

When Microsoft acquired the company behind FoxPro, there were rumors that they had buyed the company only to have access to Rushmore so they could add this also to their Access database product.

Aston Tate was lagging, Borland had Office products in DOS well ahead in functionality compared to MS at lower price point (I still have Quattro - much better than Multiplan) and was challenging in Compiler arena as well (MS lured Turbo Pascal dev later into Dotnet...) Borland aquiring Ashton Tate coupled the risc of dBase getting competetive (or at least more functional/less bugridden) with the risc of it being offered at Borland price points. MS bought the better product, but the real market share changer was Windows 3.1 / 95 integration of Fox and new Office products. Lotus, Wordperfect, Borland, withered away the same as Clipper, dbMan, Arago, which never regained their DOS market share when they had Windoze products.

regards
thomas

mainhatten · Post by **mainhatten** » Fri Jan 31, 2020 7:21 pm

wriedmann wrote:Unfortunately your time is limited (as ours is too....)

Wolfgang,
I fully understand preference for having sources to check if something surprizing occurs. But if you really have a need for better filtering, why not use Advantage Local Server ? I know for certain that they have not only memory bitmaps, but they also support the vfp binary index type - which makes sense only if they operate Rushmore-similar in memory as well. That was the source of my opinion that current other xBase variants also had such tech.
Crippling their RDDs intentionally by removing that functionality is hard to imagine...

As we are on RDD topic: I read part of the sources, googled for it, but found little, probably best was:
https://vivaclipper.wordpress.com/2013/ ... hitecture/
At least this old fox is not avoiding new topics, and RDD seem not only to be important in Clipper heritage, but also a good concept to master. When I read in VO centered threads I realize quite a bit of vocabulary and "automatic" concepts are missing on my side - can you (or one of the other Clipper/Vulcan/VO veterans) provide a link list of resources ? The thing a newbie gets when working on your sources first time fresh from university ?
If it includes links to a few well written (=most of it text, not "cheap" screen shots) door stopper books en vogue in the 90ies, all the better, as I already read too much on tablet

tia
thomas

wriedmann · Post by **wriedmann** » Fri Jan 31, 2020 9:06 pm

Hi Thomas,
I'm already using ADS in some projects where I either have a lot of users (largest site has an ADS license with 120 users) or the need of fast filtering (smaller sites with 5 to 10 ADS connection licenses).
I'm using the ADS local server on my development machine, but for customer projects I don't think it is a good solution, first because it is limited by connection numbers, and second, it does not work in remote desktop sessions (even when I connect to my development machine in remote desktop, ADS local server does not work).
Regarding the VO/Clipper vocabulary: sometimes I feel the same when reading messages from your VP people <g>.
Unfortunately I don't know any source to that vocabulary - maybe because it is non written anywhere, but was established over time.... Please feel free to ask if there is something you don't understand or what is not clear to you, of course also in the German forum, if you like.
Wolfgang
P.S, maybe I should add then a few pages with your questions in the docs.xsharp.it wiki

wriedmann · Post by **wriedmann** » Sat Feb 01, 2020 8:22 am

Hi Thomas,
let me give a few terms you may need explanations:
RDD
DataServer/DBServer
DataWindow/DialogWindow
Codeblock
Macrocompiler
GUI Classes
SQLSelect
ArrayServer
Please add/remove it you wish.
Wolfgang

mainhatten · Post by **mainhatten** » Sat Feb 01, 2020 11:07 am

Hi Wolfgang

wriedmann wrote:I'm already using ADS in some projects where I either have a lot of users (largest site has an ADS license with 120 users) or the need of fast filtering (smaller sites with 5 to 10 ADS connection licenses).

So on technological side problem is solved

I'm using the ADS local server on my development machine, but for customer projects I don't think it is a good solution, first because it is limited by connection numbers,

just not inside the RDD currently written and avalable for xSharp.

and second, it does not work in remote desktop sessions (even when I connect to my development machine in remote desktop, ADS local server does not work).

That does ring a faint memory bell... From memory after a vendor session introducing ADS version wth support for index type "binary", there is a programmatic check due to licensing reasons. ADS does not want local server to be used where "normally" remote server is needed. Gist from back then was,
a) even on a production VMs it would be legal to use local server, as long as no shared access to a dbf from different VMs was done, "use again" from same machine allowed.
b) when developing, simulating several real PC accessing with local server is allowed due to dev exception, although hoped by ADS dev that a remote server is used in production. Same would probably cover a dev accessing production server for purely debug reasons.
In those cases the technical block could be eliminated, so get in touch with SAP sales rep, ask if above description is still true and if so, let them walk you the steps necessary to allow at least RDS on your dev machine, as you can point to.applications you developed resulting in sales for them.
If single machine access/use is still allowed and your need is pressing, a Rube-Goldberg approach could be to access remote DBF via vfp OleDB (similar to cursoradapter, implemented with ADO.NET existing objects), pipe result into local dbf, access "as cursor" via ADS RDD with a few enhancements (things needed for Buffering, Update Conflict handling, Tableupdate) and persist to remote dbf via vfp-OleDB based Dotnet-"Cursoradapter".

Regarding the VO/Clipper vocabulary: sometimes I feel the same when reading messages from your VP people <g>.
Unfortunately I don't know any source to that vocabulary - maybe because it is non written anywhere, but was established over time.... Please feel free to ask if there is something you don't understand or what is not clear to you, of course also in the German forum, if you like.
P.S, maybe I should add then a few pages with your questions in the docs.xsharp.it wiki

"Resource pages" both for VO/Clipper and Vfp on the Wiki sound like a good idea, perhaps with a mention on info for newbies here plus reminder every release or at least half year....

regards
thomas

mainhatten · Post by **mainhatten** » Sat Feb 01, 2020 12:36 pm

a few terms needing more explanations: (not quoted, so others can enhance as well)
RDD - this in depth, methods, different ones available, expiriences...
SQLRDD here with expiriences or even better: benchmarks
DataServer/DBServer
DataWindow/DialogWindow
Macrocompiler as we also have a Macrocompiler, best if you know both and can define differences
Codeblock
GUI Classes as we also have a GUI classes, best if you know both and can define differences
SQLSelect
ArrayServer
AEF
Best practice Locking/Update conflict handling, if something above manual RLock/Write/Unlock exists
Architecture models considered good or best practice
Typical usage/role/toolsupport/architecture for Web applications (in contrast to xBase++ server scripts, FoxInCloud, West Wind if knowledge on multiple approaches exists)
History Clipper/VO/Harbour/xHarbour/Vulcan, more emphasis on "can I use ideas sources", less on "Who did what"
external well written Tools
web sites with good info

Main menu