good progress on Unix, bad news, good news, bad news

Got the latest version of CS2Wiki running on a Solaris 10 system today. Still there are a few niggling details. But no show stoppers. I'll release them soon. The main thing is that you need to install the Berkeley DB version 4.3 from sleepycat--it is included in the Windows release of Python, but not in the source version that you install on a Unix system.

The bad news is that the hosting company I found was really only just a web hosting company, though they say they do hosting.

Now for me the really good news is that I started using the CS2Wiki at work today, both for managing a program that we offer to small ISVs and to handle the extreme amount of email we all get. So I'm adding a LOT of content to the ark, and running on Unix at that!

I managed to crash the Wiki twice, but haven't had much time to look into the tracebacks. And that's the bad news.

But to end on a good note, I expect I'll be using the ark now every day at work for an hour or so, converting a pile of emails into easily managed knowledge structures. :D

progress on Unix, hosting

Well, I finally got CompStrm running on Unix. Turned out to be a line termination problem.

Now it will still take me a release or 3 before I can say done, as I am developing on Windows, releasing, and then testing on Unix. So don't get too excited just yet.

I've also found a host that is willing to install CompStrm for me, http://www.21centuryhosts.com/ The only catch is that I pay for installation and I'll probably want to upgrade from time to time.

Descriptors

Norm and I have been exchanging a series of emails dealing with how descriptors should be implemented in the CS2Wiki, and I'd like to describe what we ended up with.

First, the CS1Wiki already has descriptors to control the sorting of LSecs and Pages (or not). Color is also a descriptor, though its currently accessible from the cls display. However, these descriptors are implemented as special TKCS properties, each with its own set of Wiki commands, and we need a way to implement a multitude of descriptors, often type specific. But note here that descriptors, like color, can be inherited.

The temptation is to use headers for descriptors, but that's not a good idea. Headers are data and are the meat of users, along with other text content. Descriptors deal with behavior and are configured--a very different activity and likely not something most users will be concerned with.

So the descriptors which apply to a particular Role will need to be put in that Role's descriptor dictionary, much like the way we handle contextual names. And we'll likely have a dsc command that puts you into descriptor mode, and a set of commands which only work in that mode.

I expect also that there will be a lot of Roles which have (mostly) the same descriptors. So we should be able to define a Role which serves as a Descriptor Unit for other Roles. Now in this case, the headers (and LSecs) in the Descriptor Role will be the descriptors for other Roles.

Now when you combine a Descriptor Role together with a Role's own descriptors and the descriptors it inherits from its applicative context, and you start to have a moderately rich system.

Progress on the Calendar Spec

The calendar we will use in CS2Wiki was developed by Norm Kashdan and Frank Frazier: http://compstrm.sourceforge.net/calendar.html

Norm has reviewed the spec that I wrote and I've consequently made a few corrections, including the addition of cycles. I also removed bad name--the month pages were naming week pages.

The next step will be to create some preliminary scripts which generate a partial year.

headers and LEnts

While working on the Relations1 and Phonebook1 examples, I realized that I couldn't integrate them the way I wanted to without name/value pairs. So I've added email-like headers to the text content of Roles. One implication of using email-like headers is that the names are not case-sensitive. (This will also make it easier to add email access in the CS5Wiki.)

Of course, that's not quite good enough. I've got to add the TKI support for accessing those headers before I can use them in the TKI examples.

But now that I've got those headers, implementing a first draft of LEnts may be an easy next step.

There are several ways that LEnts differ from LSecs:

• LSecs can hold Remarks, LEnts and other LSecs, while LEnts can only hold Remarks;
• The name of an LSec is user-assigned, while the name of a LEnt is generated;
• When listing LEnts, the headers are displayed, while only the names of the LSecs are listed; and
• Cabinets, Folders, Pages and LSecs can have LSecs, while only LSecs can have LEnts.

LEnts don't really start to get interesting until we can add some descriptor data to identify which headers are to be used when listing LEnts. (This descriptor data will likely vary by type of LSec.)

a new design for the TKI examples

ToDo

In the ToDo example, stuff becomes a Page and the items to be done become LSecs under that page.

Relations

In Relations, each person becomes a folder. Names are then used to define the relationships.

The only catch here is that to view the inverse relationships, you need to use the cls command. Clearly there is a need for a custom display capability specific to an application.

PhoneBook

In this example, each person becomes a folder, and each phone becomes a page. And since a page can be in more than one folder, there is no problem with more than one person "sharing" a phone.

In addition, the phones can be named within a person's folder. Typical names would be MobilePhone, HomePhone and OfficePhone.

Application Integration

Note that because common (CS1Wiki) data structures are used, it is now trivial to integrate the various applications. The folders used in PhoneBook can be the same folders used in Relations. And one or more todo pages can be added to the folders as well.

Now the intent is still to support types, which allows for custom commands and displays (G4). But note just how powerful naming can be.

An unexpected result

I started thinking about the TKI examples and realized that the Wiki structures are easily rich enough to support them without further augmentation.

So all I really need to do is (1) refactor the Wiki code to produce a richer API and (2) build TKI commands around the new API.

And then, of course, completely revamp the TKI documentation. :-(

Now if/when I create a means of running TKI scripts from the Wiki, it will start to get pretty interesting. But meanwhile, this is the perfect rampup to prepare for writing the Calendar generation script.

Meanwhile, here's the updated script for creating the todo1 sample data:

cd /WellKnownDirectories/RootDataSet/Data
set wkn {TKI Sample Applications}
mkCabinet
cd %ip%
set wkn stuff
mkPage
cd %ip%
set wkn {wash the car}
mkLSec
set wkn {vaccum the house}
mkLSec
set wkn {do the dishes}
mkLSec

The neat part here is that all the data is accessible via the Wiki.

the fun is done, now what?

Well, now I can create all kinds of roles using a TKI script. Enough that I can start converting the TKI examples. Now, do I keep playing with TKI or start working on the port again?

(I only work on the port on weekday mornings. The examples I can work on on the weekend.)

Oh, and there's a whole lot more TKI scripts I could write, like name and mv. But I think I'll work on them "as needed". And I'll definately need a bunch when working on the calendar generation script.

definately past the worst

If you want to follow along, I've been updating the capability
pages listed at http://compstrm.sourceforge.net/compstrm2/tkcs/capabilities.html

Things are going well enough that I'm tempted to delay further
work on the Unix port... I'm on a roll! (For a change.)

I'm also thinking that I'll need to refactor a good part of
the Wiki code (a good thing) if I'm going to use TKI scripts to
generate calendars.

past the worst

I have not yet reviewed the various tki scripts for creating DataSets, but I've updated and validated all the other low-level scripts. Guess I'm over the worst of it.

It was nice having a block of time to dig into this. A nice Sunday afternoon project. ;-)

CS2Wiki--off to a slow start

Well, I've got the TKI navigational commands updated anway. Still, there's a lot of out-of-date code that needs to be addressed before I look at the examples.

CS2Wiki Base Release

OK, rather than take a chance at destabalizing the CS1Wiki, I've moved the remaining todo items to the head of the list for the CS2Wiki. This means that CS1Wiki will not be usable on anything except Windows. :-(

so much to code, so little time

I'm looking at the TKI support for the CS1Wiki structures, and finding that the code is really dated. There's a lot of work to be done here, starting with some navigational commands, before I even get to updating the examples.

The CS1Wiki also makes no direct use of TKI, except that TKI is used to define the its properties--which is vital for things like garbage collection. TKI also plays a role in dump/load and debugging.

So, while it has no direct user impact, I need to invest the effort in updating the TKI scripts before proceeding. This is especially important for G3.

new user, more bug reports--a confession

I'll admidt it--I am not in the habit of using my own tool, CS1Wiki. (Did you know, CS1Wiki has no google matches! Sounds like a good name choice.)

I'm taking a web course and was taking hand notes, as is my habit. Half way through the course, it struck me--use CS1Wiki. And it worked very nicely.

A Wiki is not normally a good tool for note taking. With structure implemented in content and names, its not very agile. CS1Wiki changes all that. You can easily reorganize things on the fly as you add content.

And by using structure to isolate what you're working on, I expect that you could even use a public CS1Wiki for note taking. You could even share and colaborate on the notes during the meeting.

As for the bugs, I found <> displayed as < and >. And there was no way to escape a { or }. Gotta fix that!

An Introduction to Rolonics

Took some time off this morning to do some writing. I'll get back to porting next week.

The key points I covered today are Context and Ledgering vs Journaling.

http://compstrm.sourceforge.net/rolonics101.html

slow porting progress

Spent some time today deep inside tki/tkcs/tks. I was just trying to run the tki help command. Fixed the obvious bug that I introduced eariler, but still problems. Looks like descriptors have not been loaded into the database.

I'm thinking at this point that there must be a serious (Unix specific) flaw in the code which initializes an empty database.

some small progress on the port to Unix

Been pretty busy at work lately, prepairing for year end. And last weekend I wasn't feeling well. But finally things are coming together again and I was able to spend a short time this AM IST (Indian Standard Time) working on the Unix port of CompStrm-1.

I ran into a bug that was a bit of an anoyance--the stop command, when entered in the lineclient, failed to completely shut down the server. So I focused on that.

I found that by reorganizing lineserver.py, the takedown completed. And that's very handy when doing a lot of test/edit/test.

Now there are more significant bugs remaining--I can't even do a tki dir command from the client. But at least I'm making progress.

tuning the CS1-Wiki

The CompStrm-1 Wiki is in a pecular phase of development. Within the constraits of G1 (a number of issues intentionally not being addressed until a later generation), it works. Now its being tuned.

Norm is working with the CS1-Wiki, developing usage patterns which fit well with its capabilities. But some things are just too tough or obscure to be workable. Case in point, promote and demote were found to be just to hard to use when dealing with populated structures.

There is also a matter of balance. Eariler, I had allowed references to any name or child of any visible topic. But it wasn't always obvious then how a reference would be resolved. Now the displays have all been enhanced, and there is even a ref command which shows how each reference will be resolved.

And now we've got the reverse problem. With the restricted referencing capability (we had always allowed references to children of visible topics in eariler Arks), many more includes are needed. So I'm going to revert the referencing rule to make contexts easier to configure.

So just what is Norm trying to do with the Ark? The Ark is a tool for creating languages. Within each context there is a different set terms (WikiNames) that can be used. Using structure, includes and names, you can make the appropriate terms referencable for each such context (effectively defining a language for each context). Then you can compose text content and have at hand all the terms you need to weave a very fine knowledge net.

displaying helpful names

I hate absolute path names. They don't show how the current context is connected to an item. I've studiously avoided them, and finally that effort has borne fruit.

When CS1W displays a reference, you want to know the (shortest) relative path from the current (applicative) context to that item. Often that relative path will include a contextual name, and you may also want to know what the real name of the item is. So when a contextual name is used, a relative path name which does not use any contextual names is also displayed.

This is a big step up, and was implemented in 1-0522f. This change should make it much easier for a user to grok the contents of the Ark and all the relationships defined between the various items it holds.

changing G2

I really want to add true Ledger Entries, so I've made it central to G2, pushing everything except the calendar generator into a separate generation. This will give us improved list handling and a good way to work with spreadsheet data.

(Think about setting up the applicative context so that the visible ledger entries compose an interesting list that can be exported to a spreadsheet.)

With the new CompStrm-2, we begin to realize more of the rolonic capabilities. Its a lot more than a Wiki--that's just the user interface.

I've updated the CompStrm to reflect the new G2: http://compstrm.sourceforge.net/

another piece in the porting puzzle

I'm happy to report some (limited) progress in porting to Unix: I found Danny Yoo's _Getch at http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/134892

So now it looks like I just need to slug away for a while and get it all to work. I've been thinking about going back to the CompStrm tutorials and revalidating them on Unix.

promote/demote inadequate

When working with populated structures, promote and demote simply are not powerful enough. There are two ways to enhance these commands (for G1!):

1. Require an argument, the name of a child or wildcard for same. The command then applies to the selected children, not to the current item.
2. Make promote/demote recursive. Exception: LedgerEntries are not automaticly promoted nor demoted.

Its important to keep the Wiki agile, which means having commands which are powerful while not taking control away from the user. And the (limited) experience we've had so far with the CompStrm-1 Wiki is that promoting/demoting is important when attempting new usage patterns. These commands need to compensate for the rather ridged structures we've set up (Cabinet, Drawer, etc.).

lsecs vs lents

Had a great IM discussion with Norm. I had never understood the need for both lsecs (ledger sections) and lents (ledger entries). Lsecs could have nested lsecs or lents. But why differentiate?

Well, eariler versions of the Ark never supported lsecs, so it wasn't much of an issue. To me, the distinction is in naming. Lsecs have names, lents have headlines. Names can be used in pathnames, headlines can not. Headlines can, in turn, contain names of other items. Headlines are more like 1-liners.

So CompStrm-1 contains only lsecs, but named as LedgerEntries. This will change in G2. There are lots of things that can be done with lents. We will however need a special form of include for including lents--it would have both a pathname and a headline filter.

One common use of lents is to use them to hold coma delineated data, and then export/publish them to a spreadsheet. And, of course, that means composing via the incLent a set of visible lents to be published.

G1 is very insular--its hard to get data in and out of it. That wasn't true of eariler arks and it will not be true of later generations. But this all takes time, and I got a day job--a good one.