More work needed on cc- commands

A "ccc *" just isn't too useful. And that's what you get now when you "ccc".

What you really want is the ancestor cabinet.

And for a "ccd", you want either the ancestor drawer or a picklist of the ancestor drawers. That would be much more handy.

The above was my expectation when I released the changes to these commands, and Norm has concurred. So there's still a bit of wiki code to write.

Like I said, the code probably isn't finalized yet. There might even be some bug reports!

progress in porting to Unix

Found a few bugs while porting. Now I just need to track down all the \'s in the wiki code. Shouldn't take too long.

I'm porting to Solaris 10 with bsddb 4.3, twisted 1.3 and Python 2.4. I've gotten as far as the welcome page. Which means that the database initialization has completed. Things are looking good.

The tki client will pose an additional porting complication, but I'm not worrying about that right now.

Wrapping up CompStrm-1

Having moved the calendaring system to G2, there's not a lot left to do in CompStrm-1. The Wiki code is done, if not finalized. And I'm turning my attention to porting and general cleanup.

restricting the use of wildcards

I've been cleaning up the cls display, adding selections and showing how includes are resolved. In the process I was struck by the obvious: citations don't work when wild cards are used.

The citation process is pretty simple--calculate every possible pathname and check for references. (Well, its more complicated than that, but that is basicly how it works.)

Now if we allow wildcards in references, we can no longer check for exact matches, but must now do a linear scan of every reference. Hey! That's just not scalable!

So there are a few places where I will no longer allow wildcards (in some future release):

• includes
• excludes
• WikiNames in text content.

partness

A rolon is both a part of a greater whole and a whole which has parts.

There are two kinds of metadata: classifier and descriptor. The classifier is partness data--it describes how this rolon works as a part in a greater whole. The descriptor data is wholeness data--it describes how the parts of the rolon work together to form the whole which is the rolon.

Descriptors will be in CompStrm-3, when we introduce types of rolons. In Compstrm-1, we've been focusing on classifiers.

Classifier data is not needed when using an Ark. But it is needed when restructuring/configuring an Ark. To this end, the classifier data (and its inversions) have been moved to a separate display.

What has not yet been addressed are the classifier commands. To make good use of these commands, you should be viewing the classifer data (before and after running the command). I expect that I'll be introducing some modality, so that these commands can only be run after the cls command and so that these commands display the updated classifier data on completion.

In effect then, the classifier becomes a distinct aspect of each rolon, with its own display and command set. And we have uncluttered the normal display for the normal user, hiding the classifier data and related commands behind the cls command. This fits the overall approach of structuring information so that you see what you need when and only when it is appropriate.

(Guess you could say that the CompStrm-1 Wiki alows you to make things easier by avoiding information overload.)

Why can't I arrange the folders?

References to Pages and LedgerEntries are kept in lists, so they can be ordered by the user.

Lists are implemented as documents, with supplimental indexes. This means that every time there is a new entry, an old entry is removed, or an order change, an entirely new document is added to the database.

So lists, if they are long and frequently updated, can eat a lot of disk space. My thought was that a drawer is often updated and would frequently contain a lot of folders. Of course, there is the added (minor?) complication that if we change it now, we also break beta.

I'll admidt that there aren't too many areas where I worry about disk space. Mostly I trade disk space for fast access. But in this case, it seemed best to avoid such an excessive use of disk capacity.

Progress on Porting to Unix

I admidt, I'm spoiled by the pre-packaged releases available for Windows. Like having bsddb included in the download for Python.

Thomas and I finally realized that there was no BSDDB software installed on my Solaris 10 machine. That was easily fixed. But the one step forward, two back shuffle continues.

CompStrm is Robust!

Having some experience now with various failure modes, overall I am quite happy.

Much of the source of this robustness is because deletion and cleanup of datastructures is not under application control, but rather is governed by the metadata associated with various properties. This means that, when an application bug makes some odd structures which the application logic is not prepared to handle, cleaning up the mess is just a de command away.

In general, its much easier to get the metadata declarations correct than to debug the (often dynamic) business logic.

running on Unix, almost

Thomas (a dear friend and an ex-employee from 3 years past) and I have been working on getting CompStrm to run on Solaris 10. Facing a few small difficulties, but I think we're close.

Part of the problem is that the Python interface to bsddb supports the newer bsddb features on Windows, while the Unix version that shipped with Python 2.3.3 does not.

The first thing we will do is to get wikiServer running on Solaris/Unix. Getting the TKI scripting client running will be later--I need to work up alternative code to mscrt for reading a character at a time.

citations vs classifier citations

Given a new display for classifiers (like name, include, exclude), I'm wondering if the references made by these classifiers should be displayed with other references on the cit display (named, included, excluded) or should I move them to the classifier display as well? I suspect the latter, though I could argue for either.

updated todo

Lots of feedback from Norm today. He's found a number of weak spots, like the inability to reorganize nested LedgerEntries. So I've updated the list of things pending on CompStrm1 (my todo list) on the main page: http://compstrm.sourceforge.net/

restricting scope: the local classifier

In general, if X has a Y, then Y is visible for all children and other decendents of X. Further, Y is visible to any item which imports X.

The exception now is Remarks. If X has a Remark, R, then R is visible only to X and to Remarks on X. By default, you could say that Remarks are local.

Lets say we want a Drawer to be local. Then it would not be visible when the Cabinet it is in is imported--in all other cases, normal visibility rules would apply.

There is a granularity ordering of items: Ark, Cabinet, Drawer, Folder, Page, Ledger and Remark. It makes no sense to talk about local Arks and Cabinets, and we've already looked at Drawer and Remark. Lets consider a local Page...

A local Page will not be visible when the container it is in (Folder, Drawer, Cabinet) is imported. A local Page will be visible to the Pages, Ledgers and Remarks that are in the same container. But a local Page in a Drawer will not be visible to the Folders in that Drawer.

Now marking a ledger as local is likely much more helpful than saying that nested LedgerEntries are local. But perhaps we could mark Remarks and nested LedgerEntries as local by default.

questioning visibility of nested ledgers

Let me begin by noting that Remarks are not visible except when the current context is either a Remark or the item being remarked on.

Nested LedgerEntries are, with the latest release, quite visible. When on Page P in Folder F, all the nested LedgerEntries of P and F are now visible. Is this right?

Perhaps when on Page P, only the nested LedgerEntries of page P should be visible, together with just the top-level LedgerEntries of Folder F.

Comment welcome!

thoughts on Ledgers

LedgerEntries can be deeply nested, unlike other structures in the CompStrm-1 Wiki. They are also a pure tree again, unlike other structures. Further, the ledger (tree) of a page or other item is a fairly limited domain.

The current problem with LedgerEntries is that they are too granular. I'm proposing then two changes:

1. A flat namespace. "cc LedgerEntryName" should be enough to navigate to any LedgerEntry that is part of a Page. And the l command should list all nested ledgers--they should all be visible.
2. The display of the Ledger on a Page or other item should list the entire tree of LedgerEntries, nicely indented to reflect how they are structured.

On thing I'm always saying is that the rules are different at different levels of granularity.

value of names

Well, names have been released, though I expect to turn up a few bugs yet.

The real value of names, I'm thinking, is that it allows a degree of indirectness. And that is always useful.

For example, if you have several versions of something (bookv1, bookv2, etc) and you want to have a name that refers to the released version, and another which refers to the draft version.
name releasedBook bookv1
name draftBook bookv2
Now when you release a new version, you can just change the names.

tenatively beta, names forshadow classifiers

With the Wiki moderately stable and the new data migration proceedures released, we are now tenatively beta. And I'm starting to work on magic names.

If this were CompStrm-3, I would create a type of classifier for naming things. But as it is, I'm just using a dictionary of name/uuid pairs. So it looks like the naming code will need to be redone later. When it is implemented as a full-blown classifier, we will be able to attach additional attributes, like if the name is usable from other contexts.

Names then are a forshadowing of things to come in CompStrm-3, implemented as a special form.

stable? Magic Names

Norm's pounding away on the 1-0519b release as I write. So far, its looking good.

Being down with the flu, going back to work tomorrow and with my wife gone until Saturday, its unlikely that I will be doing much until the weekend. And then its time for magic names.

Magic names are just names that can be dynamicly assigned in one context to reference some other item. So you can have a name like ToDay which gets assigned to a page for the current day. Kinda like Unix links.

The fun will be in the code--I'll need to refactor a large part of the wiki.

not the best of times

Took off from work today--fever. My wife, Rupali, is visiting home this week. And Norm keeps finding bugs in my code. ;-)

The code has undergone some substantial changes, so its not too supprising a few things were missed. But I'd like to get these things cleaned up before doing magic names, as that too is a substantial change.

Currently Norm is reporting difficulties with the de and chnm commands.

What's left to CompStrm1?

There remains a lot to be done.

1. data migration
2. fix TKCS examples
3. magic names
4. selections
5. promotions/demotions

Another month? ;-(

color and Applicitive Context revisited

If you use the color command, colors will now start popping out almost everywhere. That's because anywhere there is a table being displayed with one item per row, the rows take on the item's color. So stick to pastels, they mix and match better. ;-)

Applicitive Context has also been revisited, again. The new corder command gives you control over the applicitive order of the collections that an item is in. All in all, now you have pretty good control over the ordering of the Applicitive Context.

I'm also happy to note that a number of bugs have been found/fixed, especially regarding remark visibility.

Applicative Context as Implemented

There's a new command, ac, which displays the current Applicative Context. It simply lists the includes and ancestors of the current item and plays a key role in name resolution. Not to exciting, but it is the order which is important. When there is duplication of names, sometimes the first item with that name is used (i.e. when the name is an argument to a command). So the order of items displayed by the ac command is important.

Another new command, iorder, lets you order the includes. This then gives you some control over the ordering of the Applicative Context.

Weekend Plans for CompStrm1

Well, I was planning on doing my best to finish up CompStrm1 this weekend. But just now I'm not feeling my best. So we'll see.

My current emphases is:

1. finish everything relating to name resolution.
2. add migration proceedures (But CompStrm1 data will still not be able to migrate to CompStrm2.
3. some minor polishing.
4. toss just about everything else (like moves between cabinets, making copies) over the fense into CompStrm2.

I really need to move on to CompStrm2, as too many things are being held up by the poor journaling capabilities of CompStrm1--and this I DON'T want to do as part of CompStrm1.

I'm thinking that CompStrm1 will be good for demonstrations of the potential of CompStrm, and can also be used as a Wiki for things like documenting CompStrm. But journaling is a central princible in Rolonics and I really need to revamp the CompStrm Journal capabilities to be able to reap the benifits of modeling Rolonic Theory.

wysiwyg referencability

With referencability extended to all the children of an item, you can get a lot of unexpected name collision which is not apparent when you run the topics command.

When I implement the enhanced inc command, I am going to try restricting referencability to items which are visibile. You will still be able to reference the children of visible items with item/child name notation (a qualified name) and grandchildren with the item/child/grandchild notation.

This should make for fewer suprises. And that's a good thing.

I'll also note that this reduction in the size of the referencable namespace will also speed thing up a bit. And that can be important when displaying a long page with a lot of references, as this namespace is processed against every reference for proper coloring.

applicative context

Looks like the next big push is going to be on applicative context. To date, its been rather weak compared to previous work.

Applicative context is not an issue in a flat namespace, like a Wiki.

And applicative context need not be a stream when names are unique.

Yes, applicative context is about name resolution. (You could say its about giving meaning to words.)

In the ark, so far, all the drawers in the current cabinet are always visible, the folders in the current drawer are always visible, the pages in the current folder are always visible and the ledger entries and remarks in the current role are always visible. But what if we want to make the pages in another folder visible too? Currently we are limited, 'cause there's only so much you can/want to do with structural context and the include is kinda weak. So we're going to make includes much more powerful.

The intent is that when you include something, all its "children" also become visible.

Oh, so much power has its consequences. More things visible, then even more things we can resolve a name to. Suddenly the order of the includes becomes important. So we will allow includes to be ordered.

What remains for G1

Here's what I see remaining to be done in the CompStrm1 Wiki:

1. Backup/Restore and Data Migration
2. promotion/demotion
3. move
4. copy (the Rolonic term is posting)
5. enhanced formatting
6. selections (Norm's pick list)

Seems like a short enough list.

color and comments

I've had some comments from Norm on 1-0517b which are worth repeating here. The first is on color and the second is on LedgerEntries and Remarks.

Norm suggests that each cabinet should have its own color, rather than each usage. You could then set the color (a color command?). I'm inclinde to go with this request.

Norm is also finding the flexibility of having Remarks on LedgerEntries and Remarks on Remarks and LedgerEntries on Remarks a bit much. OK. Lets make it very restrictive then and add flexibility only as requested. We can initially make Remarks flat--just a stream as it were--by disallowing Remarks on Remarks and disallowing LedgerEntries as part of a Remark.

Hey, you can always build a structure elsewhere and just reference it in a remark! ;-)