Finished TheBasics

http://agilewiki.org/aw3/AwServlet?cmd=rid%20e9ab103b0e9a63d36dfd11245be870b9

Take the above link to ThePasics, the first page in an updated user guide that I'm working on. Let me know what you think.

:-)

Bill

curly brace notation proposal for relations

I'm currently thinking of the following:

{^Mother} would display the link Mother and link the current topic to the mother of that topic.

{label$^Mother} is the same, but with a label.

{label$John^Mother} is a link to John's mother, where John is any topic name or pathname.

(Note that I used $ above in place of vertical bar, which this blog does not seem to want to publish.)

Bill

Google ads on this blog?

I'd love to promote AgileWiki via something like google ads. But it takes money. The obvious thing then is to make a bit of money through Google ads, selling space on this blog and at agilewiki.org. Then use that money to pay for ads promoting AgileWiki.

Any comments? Seems like the logical thing. Especially as I'm working on AgileWiki on a full-time basis now.

Comments?

Bill

Using inferences in name resolution

Yesterday I was working on /UserGuide and was about to cover the infoset command when I realized that it has a bug--it is not displaying references--which are important when working with facet classifiers. Gotta fix that.

But this morning I was struck by the obvious. Semantic relations should be used in resolving names. If Fred's Mother is Sally, then a reference to Mother should resolve to Sally. Further, if it can be infered that Fred's Mother is Sally, a reference to Mother should also be resolved to Sally. It is so obvious. And useful. This I've gotta work on.

Previously I was thinking of applying references to the topics or names command and decided that it was just too difficult. But applying references to name resolution should work quite nicely.

Bill

back on track

I've fixed a bug in the inference rules for the example.

I've reworked the inference engine to use less memory.

And I've defined a maxRelations parameter, default is 120, though that's rather small. Still, it allows everything to run without needing to increase the default memory allocation.

Things are looking good. I also realized that the rebuild will run much faster if I give it more ram. Duh! Indeed, if I increase the block cache parameter from 500 to 2000, it will likely complete in only a few minutes. (Remember that blocks are 32KB.)

So a release today is looking very likely.

Bill

all is well for the moment

Rebuild took almost an hour. So it is time to giving the snap utility priority! The .wdb3 file is at 55.7 MB, and a zip of the log files is now 4.7MB.

The rebuild was successful, though the Deductions LSec is gone. Which is more than interesting.

Tomorrow I'll rebuild the agilewiki.org database on my PC, and look into inferencing. I suspect that we had a memory overflow or something. There were only 8 people in the infoset, but there were 45 inference rules at play. And inferencing is a geometric process. So the example may have been too large. We may simply need to put limits on the inference engine.

In any case, today has been quite an eye opener.

rebuild is too slow

The rebuild of the database has run now for over half an hour. It has become obvious that it is time to run the snap utility, which does not yet exist. Ouch!

b

real problems

Well, I fixed the first problem--I had neglected to define rules for the Son relation. So I added the rules, then added the rest of the relations, one by one. Everything worked until I added the last relation for a family of 7. Then something broke. Sue became the son of John.

Worse, the server was acting wierd. And when I restarted, I got an exception trace and a crash. So I'm now rebuilding the database--that takes time.

So all my plans gang aft aglee.

inference bugs

Having completed the documentation for the 18 inference commands, as well as updating the documentation for the view command, I've started working on a comprehensive example: http://agilewiki.org/aw3/AwServlet?cmd=rid%20d9c3e748debd60eddd9d765cc3b2f31f

Of course, it doesn't work. I've already found/fixed/checked in/deployed a number of small things found while documenting. Oh well. Looks like no release today.

Bill

A good start on the inferencing documentation

http://agilewiki.org/aw3/AwServlet?cmd=rid%2019c2875d9a326c9820d08333f3ffd503

The above URL is a good starting point for the inferencing documentation. What remains is the documentation for the 7 meta-rules for inferencing. After that you can expect a release, version 3.5, likely today.

Bill

18 commands to document

This AM I managed to test all the meta-rules. Then I deployed to http://agilewiki.org

Now I'm looking at the documentation. There are 18 commands to be documented. I guess this is not exactly a minor feature.

And while the inferencing system is working, I don't consider it finished. The problem is that there is usually going to be way too many rows created. Now I very much like the fact that new relations and qualifiers are added to the end of the table--you can then add relations one at a time and see what gets generated. But that's not really good enough.

I'm thinking that, for this to be usable, we need to be able to query the table, rather than simply display it. Hmm. We already have tag searches. The problem is that we can have multiple deductions tables in the same cabinet. So we really need some table query commands for working with large tables.

Bill

first draft of Semantic Inferencing System is now complete

Well it is done and checked in. But there is a lot that is not tested. That's next. Tomorrow!

It has been a good day. :-)

Bill

The deductions table

Time to get serious about the output of the inference engine.

The output would be rows of relations (one per row) under the Deductions rolon. The easy way would be for the infer command to destroy all existing rows and then create afresh. Not good if there are a lot of relations and the changes from one run to the next are small--a likely scenario.

Better to take a difference, delete the rows which no longer apply and then add the new rows as needed. The advantage here is not only speed, but the journal then shows the changes, and those changes might just be meaningful input for an application!

Bill

meta inference rules

We now have the following meta inference rules, which seem to have a degree of completeness:

0. (implicit)

Father(Fred,Joe) => Father(Joe)

1. Symetric(A)

A(x,y) => A(y,x)

Symetric(Brother)
Symetric(Sister)
Symetric(Sibling)

Brother(Fred,Sam) => Brother(Sam,Fred)

2. Transitive(A)

A(x,y) and A(y,z) => A(x,z)

Transitive(Brother)
Transitive(Sister)
Transitive(Sibling)

Brother(Fred,Sam) and Brother(Sam,John) => Brother(Fred,John)

3. Implies(A,B)

A(x) => B(x)
A(x,y) => B(x,y)

Implies(Father,Male)
Implies(Sister,Female)

Sister(Joe,Sue) => Female(Sue)

4. ImpliesInverse(A,B)

A(x,y) => B(y,x)

ImpliesInverse(Father,Child)

Father(Fred,Joe) => Child(Joe,Fred)

5. Chain(A,B,C)

A(x,y) and B(y,z) => C(x,z)

Chain(Father,Brother,Uncle)

Father(Fred,Joe) and Brother(Joe,JJ) => Uncle(Fred,JJ)

6. QualifiedObject(A,B,C)

A(x,y) and B(y) => C(x,y)
A(y) and B(y) => C(y)

QualifiedObject(Parent,Male,Father)

Parent(Fred,Joe) and Male(Joe) => Father(Fred,Joe)

Woops! I missed one:

7. Distributed(A,B)

A(x,y) and B(x,z) => A(zy)

Distributed(Father,Brother)

Father(Fred,Joe) and Brother(Fred,Sam) => Father(Sam,Joe)

a night off, a late morning

Been working much too hard for far too long without a break, so yesterday I picked up Elizabeth Moon's Trading in Danger. Finished the book and got to bed at 2AM (instead of 8PM). So I'm not exactly "fresh" this morning, but I am feeling quite refreshed. I really need to take a break from AW once a week. Otherwise I get stale after a few weeks and productivity suffers.

--b

enough rules!

I've now added an implied meta rule, as well as allowing a relation to be restricted in usage to only qualifiers.

I can thing of more meta-rules that will be needed, but it is getting to the point where what we need is output. So that's next.

Some more meta-rules that we will need soon:

chaining: x(a,b) & y(b,c) => z(a,c)
ex: my father's brother is my uncle.

qualifying: x(a,b) & y(b) => z(a,b)
ex: my female sibling is my sister.

Bill

Brother(Bill,Bill)

I've now added the symetric meta-rule. A problem comes up when a relation like Brother is both symetric and transitive: People end up being their own brother.

I'm thinking that some relations (equals) may allow the subject and object to be the same but others (brother) will not. This can easily be handled by a LEnt on the relation DU and a small change to the inference engine.

Bill

working, but not complete

The inference engine (invoked by the infer command) is now working with the transitive meta-rule. Currently there is no output (except diagnostics), but I'm more interested in the engine right now. I want to define a few more meta-rules and extend the engine to support them.

Bill

finally starting the infer command

Had to take care of a few more things fist:

• New DeductionsDescriptorUnit.
• New DeductionsRolon.
• New ruleReferences command lists the Rule Relation CSec references for a Relation DU.
• The du command has been moved to the navigation command group.
• Custom displays have been added to TransitiveRuleRolon and RuleRelationCSecRolon.
• Table _rrRef- has reverted back to _rr- and takes as the key the name of a relation DU.

But now I can use the crTable command to create a Deductions table. The next step then is the initialization logic in the infer command. This is where we query the infoset and add what is already known to the deductions table, while also setting up some transient data structures for use by the inference engine.

Bill

GUI Enhancement

Release 3.4.3.0 is out. Finally you can turn on all the features and all the commands are visible without scrolling up and down. That was a pain.

I'll also note that the new fieldTypes command will be very helpful when creating custom table DUs--you need to enter the field type name for each column in the table.

Also you will see that DUs are now better supported, both in the displays and the command set. That is, if you enable the Descriptors feature. :-)

Meanwhile, I still have not yet started the new inference engine. But I think it was worth the delay.

GUI Enhancement

Release 3.4.3.0 is out. Finally you can turn on all the features and all the commands are visible without scrolling up and down. That was a pain.

I'll also note that the new fieldTypes command will be very helpful when creating custom table DUs--you need to enter the field type name for each column in the table.

Also you will see that DUs are now better supported, both in the displays and the command set. That is, if you enable the Descriptors feature. :-)

Meanwhile, I still have not yet started the new inference engine. But I think it was worth the delay.

Ready to work on the new inference engine

I've now completed the framework for inference rules and implemented an inference rule that doesn't do anything.

Now the fun begins. :-)

Bill

Thinking about inference rules

Inference rules are interesting, because they are rules about relations. So a rule will have CSecs. These CSecs will all be of the same type, as they will be referencing only relation DUs. But there may be more than one CSec on a rule. Consider the implies rule, for all x: Father(x) => Parent(x). In this case there are then two variations of CSec, RuleSubject and RuleObject. So I'm thinking that there should be multiple RuleCSec DUs for a single class of RuleCSec Rolon.

Now consider the all implies rule, for all x: A(x) & B(x) => C(x). This rule would have two RuleSubject CSecs and one RuleObject CSec.

Each rule will have a RuleRolon and a RuleDU. Rules can be located anywhere, so long as they are in the infoset of the inference engine.

I've also been thinking that the output of the inference engine would be a table.

Some of the commands we'll need:

crRule - creates a rule from a ruleDU;
rules - lists the rules which operate on a given relation;
crInferenceTable - creates an inference engine table; and
infer - Update the inference table.

Of course, once we have an inference table, we'll want to have several query commands which operate against it.

progress on Semantic Relations

I've done a lot today on Semantic Relations. The first phase, construction of relations, qualifiers and related structures is almost done--one last query left.

Now I'm thinking that I need to review the code for copy. Currently copy does not support the new _sr- and _sq- properties.

Then it is on to working on inference rules and then finally the inference engine. Its all going pretty fast, but gosh there's just a whole lot of new commands, mm? But I think the end result will be quite interesting, and useful as well.

Bill

User-defined tables in 3.4.2

I've completed the tables commands, and made good progress on semantic relations as well, though the latter is still not yet usable.

Today I expect to wrap up the documentation on tables and release version 3.4.2.0.

I'll note that user defined tables is a good step towards making AgileWiki an application platform.

Bill

User defined tables

I am dropping the table type field from the form for the crTableDU command (and dropping the crDU command).

I am also planning a crTable command, where you specify which table du to use.

The scenario for custom tables then, will go something like this...

1. go to the DescriptorUnits drawer
2. do a crTableDU
3. go to the columns table in the newly created table du.
4. use the addRow command to add columns.
5. go to the topic or LSec where you want to create the table.
6. do a crTable and specify the table du which you just created.
7. use the addRow command to populate the table.

Of course, table du's are reusable--not every table has a unique set of columns.

Bill

some progress, some bugs

Made some small progress on Semantic Relations today. View now has a Semantics feature, There is now a class called SemanticRelationCSecRolon and another called SemanticRelationDuRolon. And the DescriptorUnits drawer now has a SemanticRelationDescriptorUnitDescriptorUnit. Not a lot of code, just a bit of a mess to get my head around.

Then there was the copy bug I found this morning. That took a while to track down. Then tonight I ran into what I suspect is an applicability bug on crPage. I'll fix that in the AM and then release both bug fixes, as well as the other code.

Bill

more thoughts on Semantic Relations

I always say, the obvious is the hardest thing to see. And now I've come to realize that the type of a relationship (Mother, Sister, Male) is the DU of a Relation CSec. Ah, now we are starting to see some good integration of Semantic Relations and AW. And things are becoming a bit simpler.

Rules should also be global to a Cabinet. So lets put them in a Rules Drawer, which can be created by the Cabinet's admin as needed.

I've also realized that we are still missing a crDu and crTableDu command. Perhaps I should work on that next. (And then also a crRelationDu, too). As for a crRuleDu, we may not need it--rule DUs should either be created automatically when a DescriptorUnits Drawer is created or as part of an application deployment.

Much to do. What else is new? :-)

Bill

a new design for inferencing

Having gained some experience with the inference engine, which in turn has given me much to think about, I think it best now to design something better that will more effectively integrate with AW.

What were special relations now becomes rule rolons, which are implemented as Pages under DUs. Relations then are further subdivided into qualifier CEnts, e.g. Male(Fred) is the CEnt Male=t, and relation CSecs, e.g. Mother(Fred, Ann). Also, a relationship will implicitly define a qualifier, so that Mother(Fred,Ann) => Mother(Ann). This is especially important when dealing with infosets. (Ann my be in the infoset and not Fred).

One important rule will be QualifiedRelationship:

QualifiedRelationship(Mother, Parent, Female) -->
((Mother(Fred,Ann) <=> (Female(Ann) & Parent(Fred,Ann))) &
(Mother(Sally) <=> (Female(Sally) & Parent(Sally)))).

Inference Rules

I'll note that the inference engine does in fact support rules--the metarelations are actually rules.

Consider the relationship of Mother and Parent. This does not cover everything:

KindOf(Parent, Mother)

We also want to say this:

Mother($Joe,$Sally) <-- Parent($Joe,$Sally) & Female($Sally)

We also want to say Communitive(Inverse)
and Inverse(Parent,Child), and no doubt a whole lot more, like

Son($Sally,$Joe) <-- Child($Sally,$Joe) & Male($Joe)

We can make the inference engine extensible. And then later we can add language for defining rules. But right now I am looking forward to exploring applications that sit on top of even a simple inference engine. It should give a lot more flexibility to how relationships are defined between entities.

Bill

great progress on the inference engine

I've got the inference engine working, but not yet integrated with the wiki. I've also implemented the Communitive, Transitive and KindOf meta-relations.

The next step is to make it faster by moving some of the combinatorics into the special relations where they can do early pruning.

Now this is a VERY simple inference engine. It can not, for example, handle any rules. Still it may prove useful.

Bill

thinking about semantic relations and inferencing

I've been giving this a lot of thought. The idea is to implement something very simple, but usable.

Relations

Relations would be implemented using CSecs. There will be two forms: R(x), where x is the parent of the CSec, and R(x,y), where y is referenced by the CSec. R itself would either be a topic referenced by the CSec or a special relation.

Special Relations

Some relations, especially relations about relations, would be named in a *.relations file, where a subclass of the Relations class is bound to the name. These names would begin with "_".

Special relations would include _KindOf, _Communitive and _Transitive.

So for example, if _Communitive(Brother) and _Transitive(Brother), and Brother(Sam,John) and Brother(Fred,John), then we can infer:

Brother(John,Sam), Brother(John,Fred), Brother(Sam,Fred) and Brother(Fred,Sam).

The inference Command

The inference command would operate only within the infoset of the current rolon. So, for the above example to work, Brother, John, Sam and Fred would all need to be in the current infoset.

The output of the inference command will be a (new or existing) LSec named Inferences, under the current rolon. (On completion, the Inferences LSec becomes the current rolon.)

Applications which use inferencing then can compare the latest results with eariler results to determin what has changed.

Sound simple enough? Remember that we're only intending to make it easy for an application to work with relations between topics/lsecs. Call it ai, not AI.

Why a Limited Infoset is Essential

Consider now _KindOf(Relation,Brother). Adding this to the above example, we should also be able to infer Relation(John,Sam). But note that the number of things we can infer just jumped from 4 to 10. This is why it is so important to use a reasonably small infoset.

The inference command may throw an error if the infoset is too large, or if the number of inferences is too large.

Bill

3.4.1.1 is out and now documented, starting the inference engine

In addition to reorganizing the command groups and two bug fixes, 3.4.1.1 adds 5 more commands, bringing the total up to 60. And I'm happy to report that the documentation at http://agilewiki.org is now up to date.

I've begun then to seriously think about adding semantic relations and an inference engine to the AgileWiki. Inferences will NOT be persistent, but computed at the time of a query. That adds flexibility without much complexity.

The key to making the inference engine practical is in restricting the working set, of course. And this will be done by using the infoset of the current rolon when a query is made.

No, this is not going to be a super intelligent system that can solve big problems. Rather, it can be used to solve small application configuration issues and for defining relationships between the entities to which an application is applied.

Bill

Who's on first? --A&C

Spent much of yesterday reorganizing the command groups, and then did a bit more this morning. Looks like I'm cleaning the nest before starting semantic relations.

There are now 4 commands I also want to work on:

• matchedBy,
• includedBy,
• eariler and
• later.

The first two should be pretty easy, and the last too should not be too hard either.

Now I also need to do a release soon, as the tag match classifier was broken in the last release (and likely eariler releases as well).

Bill

Nice things are happening

I've just enhanced the view command description, so that there is a link to an infoset of selected commands for each of the features. Looks great, useful, and a nice sample of the kinds of things you can do with infosets and tag matches.

Now before starting on semantic relations, I want to add two more commands, includeed and matched, which display the CSecs which include or match a given rolon. The inverted table to support this is already in place, so it is simply a matter of adding the queries.

I also want to enhance the tag match csec to support the inclusion of the infoset of the matching rolons as a configurable option.

Bill

relationships get complicated fast

Remember that a triple CSec has a relationship ("wife") and a topic (the wife).

Now that relationship will probably be of type relationship, which would be a special kind of topic, I'm guessing. And it would need some DEnts to indicate if the relations ship is transitive and/or symetric. I.E. If George is my brother then I am George's brother. And if Henry is my brother, then George is Henry's brother. --brother is both transitive and symetric.

The isa relationship will also be a fun one, especially when applied to other relationships... brother isa relative, so Henry is a relative of George.

Now, how do we do that! :-)

semantic relationships and time

Norm always says that we can change our perception of the past, and of the future. I believe this has a lot to do with semantic relationships.

I was an employee of OSF. That relationship has a specific start and end date. The relationship still exists--as an ex-employee, I am a member of the OSF Alumni group. So I am thinking that creating a CSec which defines a semantic relationship may not occur on the date the relationship started. Indeed, the entry may even be made after the end date of the relationship, as I enter information from my resume.

Now as I navigate time in AW, the CSecs as they were for those past times will change. So time then has more than one axis. There is the time of entry/correction/update as well as the effective period of the relationship. Norm calls this 2-time.

So lets then look a bit closer at a triple CSec. It would have the following:
1. the parent to which the relationship applies. (noun)
2. the rolon that the parent has the relationship with. (object).
3. the rolon defining the kind of relationship.
4. the starting time of the relationship (creation time of the CSec, by default).
5. the ending time of the relationship (indefinite by default).

Sound like fun? I think this is the beginning of the next set of work on AW. AW then is moving from ontology to semantics, at least in a small way. And I'm thinking that we can build some nice applications with this.

Bill

entities, precursors to applications

I realized this morning that before we can address applications effectively, we need to be able to define entities and collections of entities.

An entity, in AW terms, is a Topic. It could be a person, a political movement, an idea, a phone number, an address, or my cat, Sassy.

Entities of course have structure/state/ledger. And if we define Ledger Entries, then likely we can more easily process that kind of entity in an application. So for each type of entity, we'll want to define the LEnt table which describes those Ledger Entries.

But what interests me more are the relationships. I've got a pet, a residence, a wife, relatives, phones, an employer, etc., and partness is not going to be a reasonable relationship for all of these. And tagMatch is really good for relationships to streams of entities, but not for the more structural/static relationships. Perhaps include? No, that is good for namespace extension but is not really a relationship.

Well, we could abuse include a bit. Say I have an include CSec called wife. That's using a convention, which could work. But it doesn't get us all the way there.

Lets say we have a topic called "wife", and a CSec "triple" that has me, my wife, and "wife". I.E. A semantic relationship. Is that the way to go? Note that there are 4 rolons involved here.

3.4.0.0 is out!

This latest release finally includes back and forward commands--necessary as the browser back button does not work so well with forms and iframe. These commands navigate to a different context, rather than just to a different web page.

Also, with the departure of the rolonic view, the remaining (wiki) view now is much more complete, finally including the time (when navigating in time) and the DU name (when the Descriptors feature set is enabled).

So, I would say that 3.4 represents a milestone in usability. AW core features are complete.

Bill

some GUI progress

To date, we've had two views: wiki and rolonic. Some commands required the rolonic view, time navigation required the rolonic view (otherwise the time was not displaying) and the wiki view was not displaying the DescriptorUnit name when the Descriptors feature was turned on.

Now all this functionality has been moved to the wiki view, and the rolonic view has been dropped.

Should be a bit simpler to use.

Bill

Updated user guide

http://agilewiki.org/aw3/AwServlet?cmd=rid%208dfe7005e3905bb867174da3c329877c

I've spent some time rewriting the user guide (above). I do hope you find it helpful.

Bill

3.3.2.0 is out

Release 3.3.2.0 is out, with bug fixes and some GUI enhancements. See http://agilewiki.org/aw3/AwServlet?cmd=rid%20AgileWiki3 for more information.

As before, the focus will continue on improving the GUI and enhancing the documentation.

Bill