More on Rolonics

First, I'd like to point out that Norm's blog, which should be covering more on Rolonics, can be found at: http://ontologist.blogspot.com

Now what I want to do here is cover some of the rolonic terminology and its relationship to the CompStrm implementation.

At the top-level we have the Ark, which is a knowledge repository and a whole lot more. (And a whole lot more applies to everything rolonic, so I'll drop that at this point.) This would roughly corrispond to a TKCS database and an executing CompStrm web server.

Within the Ark you find Roles. Roles are organized into Realms, but a Realm is also a Role. In CompStrm, we've got DataSets which are used to organize WellKnowns, a DataSet also being a WellKnown. And that's really as far as I've gotten.

I have talked about WellKnowns holding lists of secondary WellKnowns. In Rolonics, a Role has a list (possibly empty) of Ledgers, where a Ledger may also have a list of subordinate ledgers.

The area where CompStrm is weakest is the Roles, which means my DataSets are also quite weak. A Role has four Units: Classifier, Descriptor, Ledger and Journal.

A WellKnown does have an associated page, which would be the equivalent of a Ledger Entry. But a Ledger Unit can have both Ledger Entries and Ledger Sections. (Ledger Sections are lists of other Ledger Sections and Ledger Entries; Ledger Entries being the leaves of the Ledger Unit tree.)

A WellKnown also has a Journal--something I'm nearly finished with. But within the Journal Unit, you can have both Journal Entries and Journal Sections. All I have for now are Journal Entries in a WellKnown's Journal.

Classifiers are not yet addressed very well in CompStrm, except that WellKnown names are classifiers. Classifiers deal with the relationships between Roles.

Descriptors I don't have yet, but I will soon. Descriptors describe the parts of a Role (i.e. the Ledger) as well as behavior. OK, since a DataSet differs from other WellKnowns based on its Type, I do have a little bit of descriptor data. I'll have more when I extend the type data to include user commands, instead of having the commands implemented in the web command interpreter.

That's a quick overview in the rough--more later. I'll admidt quickly that my knowledge of Rolonics is pretty shallow, though I've gotten better at implementing aspects of it over the last few years.

CompStrm releases to resume

When timezone differences are great, this can slow things down. But I am happy to report that I will soon be doing (alpha) releases of CompStrm once again. Sun Microsystems is a fine company to be working for.

Am I crazy?

Yesterday morning I must not have been at my best. The command I was having trouble with was shds, not cdds. And the command works fine. The confusion was all mine.

Anyway, I expect that I'm done until the weekend. Thursday and Friday we have an Ops review. Kind of like a retreat. We meet outside the office (but locally), start early and end late each day. Joy! A two day meeting. Well, I expect it will be more fun than that. Hope so anyway.

Credit to Norm's Rolonics

I have a dear friend, Norm, who has dedicated the last few decades of his life to the development of a theory called Rolonics. And it is on my very incomplete understanding of this theory that much of my work is directed to.

Rolonics covers more than natural systems. For example, it includes intentions as one of the time axis (who intends to do what, and when). Natural systems do not exhibit intentions. Rolonics also includes context, perspective and addresses understanding.

One of the basis of Rolonics is holonics. So we have the idea that everything is both a part of something else and a whole which also has parts. But another fundimental idea is that everything can be represented as either a structure or a stream.

Actually, Norm tells me that Rolonics contains nothing really new. He has drawn his concepts from a wide range of sources. But one of the key concepts in Rolonics is that a whole is greater than the sum of its parts. And I think that very much applies to Rolonics itself.

Now the terms that I use are often different from those of Rolonics theory, and intentionally so. This is because I am implementing something quite short of a full Rolonic system. So Norm speaks of Realms (as in realms of knowledge, but actually something far richer), and I speak of Data Sets.

I've had the privilege of working with Norm for the past 3 years and hope for the chance to work with him for another twenty.

Progress on CommandId, but...

I've now updated TKI to initialize and use a CommandID to identify all transactions resulting from the same user command.

New TKI variable, too: commandId.

Updated the documentation, and in the process found a few glitches in the tutorials relating to the root data set, Data. Started working on that when I found a bug in the cdds command. ;-(

Oh well. Tomorrow's another day.

journal processing refactored

Everywhere I check past transactions, I'm now using the same code and accessing the consolidated transactions.

Next step is to consider assigning a uuid to each command (in tki, wiki and web). If I can do that, then I can display commands rather than operations in the journal--much more user friendly--and use the last transaction time as the operation time. Makes navigating time a bit more reasonable, and without having to deal with state in the middle of an update.

Off to a slow start

Fixed the cc command. (I had left out the breaks.)

Some minor refactoring. Created functions hrow, hfont and href for generating various html constructs.

In the past, I've had event streams that could be transformed into different forms of output. That's really a great way to go. I'd like to add that capability to CompStrm when I start attaching display logic to various types of objects. Otherwise it all gets hard coded as html. ;-(

On the other hand, the tki stuff uses a very limited event stream for selections. I need to include selections in any event stream composition so we might eventually be able to enter tki commands from the web interface.

How universal?

I want CompStrm to be a powerful tool for building interesting capabilities. (Application here carries other connotations.) Just keeping every change in the database, some would argue, places a tremendous constraint on applicability. In the past I would have agreed. But things have changed a lot and keep on changing.

Data sets seem to be a pretty useful and barely constrained (non-cyclic structures only, please!) way of organizing things at the top level. Attaching a list of well knowns to a well known adds even more flexibility. Support for references, citations and qualified names adds tremendously to the whole thing. Making it all accessible as a wiki (all be it, a command-based wiki) makes it usable. But do these things constrain the capabilities as well?

Having a document (wiki page) and a list of well knowns attached to every well known seems like a constraint, but these are really optional. And it is nice to be able to attach descriptive material (i.e. the wiki page) to something--potentially a big aid to user comprehension.

I'll note that there's still a lot of flexibility by either defining new well known types or by attaching new types of files to the well known's group. What needs solving is reference/citation processing when new properties are added which may reference other well knowns.

But I think the constraints really lie in what is missing. Like a federated identity service. Always there is just too much to do.

journal thoughts

I still want to refactor the code so it is easier to process a consolidated journal--i.e. a journal which shows new and destroyed references from other well knowns, as well as changes to the group structure for which the well known is the root.

Another thought is that I should tie all the operations/transactions from the same command and show it only once using the timestamp of the last operation.

second thoughts on second class user objects

I'm thinking that I should do a very restricted implementation of second class user objects initially. Just impliment an indexed file list, where the user can maintain the ordering. I'll have enough on my hands extending visibility and referencability to include secondary objects. (You should be able to see the second class objects of the current well known and direct ancestor data sets, and reference the second class objects of all visible well knowns.) Qualified naming syntax remains the same--everything is separated by '/'.

I'm also going to need to rework dump to include anything referenced by an external property in place of following dataset item properties.

So I'll need to rework the Phonebook1 example considerably, chopping out the 2 extra datasets in the process.

After that, I can start working on references and citations. But first, I still need to fix that cc command.

garbage collection enhancements complete and working

We now have external properties. And the only protected well known is Data, now in its own directory.

Second Class User Objects

Its interesting how when you fix one thing, other things become clear.

The phones in PhoneBook1 should really be second class user objects. But todate CompStrm has not supported second class user objects. It needs to.

Now that we have external properties, it is easy to define second class user objects. These are well knowns that are not in a data set, but are still referenced by an external property. So in the case of a person in Phonebook1, the person file should be in the phonebook dataset. The phone is then "in" a person. Similar rules apply--a phone can be referenced by external properties from more than one file. For that matter, a phone file could also be in a dataset as well. No reason why a second class user object can't also serve as a first class user object.

New kind of property--external

I'm using external here in almost the same context as internal properties.

When a reference is made to a file using an internal property, the two files must be (or become) part of the same structure.

An external property always references a well known. When an external reference is destroyed, the well known is checked to see if it is subject to garbage collection. Further, a file is not subject to garbage collection if it is persistent, has a strong reference from another file in the same structure OR has an external reference.

And now we're only considering a file for garbage collection when an external reference is destroyed OR a strong reference from within the same structure is destroyed.

And the property used to identify the well knowns that are in a data set will now be an external property.

Well Knowns are NOT protected!

This morning began with a question: If a data set is deleted, what happens to the pages which are not also in another data set? Well first, these pages have an incomplete context. Second, they are accessible only via the directory structure. My conclusion is that they should be destroyed. (Which is consistent with previous implementations I've worked on.)

In looking back at the CompStrm home page, I see that there is an assumption that a well known is not subject to automatic destruction. It needs revising.

Now, does that mean that a well known is structurally part of the data sets that it is in? My answer here is no--a well known can be freely moved between data sets and may reside "in" more than one.

The Data data set should now be the only persistent well known, as it is the root in the (non-cyclic) data set bush structure. Further, the destruction of a strong property reference to a file not in the same structure should now be treated the same as if the property reference were to a member in the same structure--the referenced file, if not tagged as persistent, needs to be checked to see if it should be garbage collected.

This is an interesting refinement. A data sets is itself a well known and can be the root of a tree structure, that kind of structure being distinct from the structure of well knowns created from data sets and other well knowns.

(At this point I am very glad CompStrm is not in beta.)

lazy lazy

It is Holi, as well as Good Friday, here in India. Which is why I have a 3-day weekend.

Slept late this morning. Then a nap after lunch. A fine way to start a 3-day weekend.

OK, I did spend some time reading about the changes to SourceForge, downloading an updated PuTTY, PuTTYGen and psftp, and registering a public key (now required) for my SourceForge account. But those are pretty low-stress activities.

The web command, cc, when given no arguments, lists the topics twice. Gotta fix that. And this weekend I really want to clean up references and add the fancy citations logic. But it was nice to take some time off--I've been keeping busy lately at the office.

Happy Holi everyone!
Bill

auto time change

I've been chasing small bugs this morning. The problem is that now I always want to maintain a valid context. And when I viewed the journal of a dead page at present (latest) time, there is no valid context.

The fix is to automagicly change the time when visiting a dead page. The problem is, which time? I chose the latest time on the page's journal when the well known name matches. This works moderately well.

Unfortunately, this is not always a time when the page is in a DataSet. So it is not a complete context. Indeed, there may be no time in the page's journal when there is a complete context.

I'm thinking I need better access to the consolidated journal, as currently displayed. (All a matter of refactoring.) Then I can try for a better auto time change.

Reference and citation logic is still on the back burner.

1.3 on 2.4

I noticed that Twisted 1.3 is now out, with support for Python 2.4.

I'm going to delay the upgrade until Python 2.4.1 final is out. Then I need to make sure CompStrm works with Twisted 1.3. (Some of the integration code may not even be necessary now.)

Meanwhile, I've noticed that I've introduced an error when I was working on qualified names. Introduced a bogus assumption in paint topic, that the topic name would always be present. (Turns out it isn't when a deleted page is selected.) Gotta fix this.

qualified names are working

I did change things a bit, simplifying them for citation processing.

Given {x/y}, x must be a data set and y must be in x. Further, x must be referencable from the current context.

One advantage here is that you can identify a page in a particular data set. With the more powerful notations, this kind of qualification was not working.

And that's it for the weekend. References are out of wack at this point, as well as citation processing. They're next.

pick list done, qualified references

I've been doing a lot more refactoring than usual for me. Things are looking good. I've finished the pick list for ambiguous references and even re-accelerated visible data sets in preperation for advanced citation processing.

Its been a very good weekend. But still a ways to go. I'm thinking that qualified references are next.

Using a data set name to qualify the reference to a page means that you can identify a particular page (when there are two visible pages with the same name in different data sets).

It also allows you to make references to "distant" pages which are not in a visible data set. In this case, the data set must be referenceable (or visible), and the page must be referenceable (or visible) from the data set.

Yes, I want to allow a qualified reference like {ds/page}, where ds is a grandparent rather than a parent of page. More power, but at a cost of more complex citation processing. ;-/

Once I've got qualified names, then its on to citation processing. (Right now, the citation logic is not up to date.)

in dataset

Wherever reasonable, the links to the datasets holding a page are displayed.

And I think that's more than enough for today! Still to come: a choice page when a reference is ambiguous.

One reason why a choice page is better than just picking one and going to the journal display, is that the act of picking one changes the context, changing which pages are visible and consequently the alternatives available for an ambiguous reference.

done with journals, for the moment

As previously outlined, Journal displays are now a great deal more comprehensive. That creates a problem when the activity applies to a part of the well known--as in the case when a relation dictionary is updated. We have not yet established names for subparts. (AKA second class user objects?)

But lets wait on that, least we get too far ahead of ourselves.

One of the current weaknesses is when there is ambiguity of name reference--currently being shaded gray. Current action is to pick one and go to the journal. Rather, a special pick page should be used--the journal just shows too much data.

Also, I'm thinking that the datasets a page is a member of should generally be displayed. This info should likely also be included on a pick page as well.

Topics remaining: qualified references and advanced citation processing, specialized displays, file-specific commands, secondary user objects.

caching the visible data sets

I've been getting something of a performance hit because of the cost of repeatedly recalculating the list of visible data sets. So I'm now caching it by uuid and time, clearing the cache for each new command entered by the user.

End of problem.

Thinking about the journal display

I've tracked down a few bugs and extended the display for the Journal.

• Topic is now the first item listed.
• Command and Operation names are now both displayed
• Changes in the relationships to other well known items are also displayed.

Unfortunately, the journal display is currently restricted to transactions applied to the current page/file. That's two narrow. For example, in the relations1 example, the well known has an associated dictionary. And its transactions are not being displayed.

Further, when a page is added to a data set, the transaction currently only displays in the journal for the dataset.

So I think the transactions included in the journal display need to be broadened.

a project at Sun

Looks like I'll soon be starting a sizing project at Sun. That's where you determine the hardware needed to run a product under various loads. Should take about a month.

good progress on extended web

Now everything is a web page, including DataSets. And the topics command nicely displays the visible topics. Next is to fix the journal--it needs a lot of work.

Note that I still can't use web to create datasets or move things around. Maintenance commands are missing. And application data other than wiki pages is also missing. But navigation is working just great.

down for the count

Its been a while, but I've managed another intestinal infection. Feeling better, but not fully recovered. And its something green. (I'll spare you the remaining details.)

kinds of context

Visibility and referencability really define a large part of context, context being the environment in which high-level user interactions and low-level operations occur. And there are lots of different kinds of context.

Type context is one. Operations are typically defined by type. Instance context is another--here we have instance content providing part of the operational context.

Structural context (the inverse of scope) we have already talked about--the DataSets which hold a WellKnown place a WellKnown in a larger context.

We might also extend visible and referencable rules to include explicit context, like a Python include statement. Explicit context might be defined on a type, on an instance, on a container type or container instance. The last wiki I worked on for a client had extensive explicit context but limited structural context.

Structuring DataSets for maximum utility

First, I'll note that when a well known is referencable from a given page, then adding the name of the well known to a page means that you can click on the name and navigate directly to the desired well known.

Now suppose I have a data set containing information about various contacts. And I want to reference the contact names from ANY page. All I need to do is put the contacts data set in the "Data" data set (the root data set). The name of the contacts data set is then visible everywhere, and the contacts in that data set are referencable from any page.

To restrict the accessibility of a data set, move it from the root into a lower-level data set. At first this may appear to be a bit simple minded, but remember that well knowns (and thus data sets) can be "in" more than one data set. So there really is a lot of flexibility here.

Actually, the flexibility is almost unlimited--as long as you can restrict the contents of a data set to well knowns with the same "scope".

visibility and referencability

I am in the process now of increasing Web visibility of DataSets and other well knowns. Previously, only other well knowns in the Main data set were visible.

First, I need to define ancestors. If the current page is a DataSet, it is treated as an ancestor. Also, the data sets which the current page is in are ancestors, as well as the data sets that those data sets are in, recursively.

Visible well knowns then include both the ancestor data sets as well as the well knowns in those data sets. These should be listed when the topics command is entered.

Now, given the visible data sets (visible well knowns that are data sets), a referencable well known includes both the visible data sets and the well knowns that are in those data sets. These well knowns should be accessible using the cc command.

(Referencablility is one degree more than visibility.)

Anyway, that's what I'm working on now. Should make navigation pretty easy, even without adding qualified names.

My daughter got maried

Didn't manage to fly back for the wedding, but I had met the groom, Chris, on several occasions--they were engaged while I was in the US. A very nice person. I'm sure Genny and Chris will be happy together for many many years.

Federal (US) taxes done

I efiled my federal taxes using H&R Block. Painless and free. Tried also doing my state taxes, but it just didn't work. Too many business expenses to us PA direct filing, so I'm back to paper. Still, I feel that I'm ahead. And I was able to verify that PA had received my estimated tax payments.

The big thing now is doing my India taxes. Still need to get a PAN though.

double the tax returns

Mmm good! This time round I've got US income taxes to file as well as Indian taxes. Should prove interesting. :<()

Thoughts about citations

I had been thinking that the current structure for references would be inadequate for citation processing when names are qualified by dataset. I'm now thinking that the current structure will be more than reasonable. It is simply a matter of searching for references with every name by which a page can be referenced--that will typically be a rather short list.

Color me gray--support for concurent duplicates

Up to this point, there has been no support for duplicate pages that exist at the same time (concurent duplicates). That has now changed.

The crpage and chtnm commands no longer check for duplicates. Also, when there are concurent duplicates, the link is painted gray.

Now when clicking on a gray link (or doing a cc to a topic with concurent duplicates), you are taken to the Journal view where alternate pages are listed. Links on the Journal view back to page view are always via uuid, so there is no confusion as to which page has been selected.

Done with Wiki, almost

At this point I'm calling the Wiki stable, though when I revise the user files, I may need to revisit it.

Except as noted then, subsequent changes will be to the web access only.

Conversion to DataSets and WellKnown files milestone

Users have not yet been converted to WellKnown files, but ultimately they need to be. The good news is that Web access is working again. The bad news is that at the moment, none of the potential of DataSets and WellKnown files is being used--Web access is the same as it was before. Web can currently only access the contents of the Main dataset.

So there's still a great deal of work left.