© 2002-2003: Jan Wolter, Steve Weiss
Backtalk is designed to be able to share its conference database with Picospan or Yapp. Picospan is a conferencing program designed by Marcus Watts and distributed by NETI. It was the original software used by M-Net and The Well. It is still used today on Grex, but is no longer available due to the collapse of NETI. Yapp is a clone of Picospan written by David Thaler. Yapp 2.3 source is available as shareware. Yapp 3.0 is an extended version including a web interface. Both use file formats very similar to, but not completely compatible with Picospan.
Backtalk can be built to coexist with Picospan, Yapp, or neither. The file formats it uses will be slightly different in each case.
Backtalk-style participation files are modified tag files.
Filename Location Description .backtalk User's home directory User's personal settings id User's home directory User's login name, uid number, and gid number. Used only in configurations built with --ident=tagfile. settings Conference directory Settings for a conference.
allowhtml=1 allowgraphics=1 login=(<CENTER>\n<TABLE BORDER=4 CELLSPACING=4 CELLPADDING=8>\n<TR><TD ALIGN=CEN TER>\n \n<BR>\n<FONT SIZE=+1>Welcome to the</FONT><BR>\n<FONT SIZE=+3>TEST< /FONT><BR>\n<FONT SIZE=+1>Conferencce</FONT><P>\n\(Fairwitness: Marcus Watts\)< BR>\n \n</TD></TR></TABLE>\n</CENTER>) new_isel=(1,3,5,10,11,$) new_since=7 bgcolor=(00FFFF) regbutton=(bgn) bgimage=() highcolor=(FF1493)Note that the "login" field is all one line, broken across four lines here for readability.
Yapp and Picospan use slightly different formats here. Backtalk uses the Picospan format, unless it was configured for Yapp compatibility at compile time.
!<hl01>
/usr/local/backtalk/bbs/general2
gen_eral2:/usr/local/backtalk/bbs/general2
general1:/usr/local/backtalk/bbs/general1
oldgen_eral:/usr/local/backtalk/bbs/general1
tes_t1:/usr/local/backtalk/bbs/test1
mar_tian1:/usr/local/backtalk/bbs/martian1
mars:/usr/local/backtalk/bbs/martian1
bar_soom:/usr/local/backtalk/bbs/martian1
The second line is the directory name of the default conference. This is rarely used in Backtalk's web interfaces, but it is in command-line interfaces.
The remaining lines each consist of a conference name, a colon, and a conference directory. Conference names usually have an underscore inserted in them somewhere. When a user types the name of a conference, he may abbreviate it by leaving off letters after the underscore. Thus, in the sample above, you could join version 2 of the general conference by typing ``gen,'' ``gene,'' ``gener,'' ``genera,'' ``general,'' or ``general2.'' To join version 1, you'd have to type ``general1'' however, because there is no underscore in the name so no abbreviations are allowed. Note that there can be more than one line for the same conference. Thus, in the sample above, we can also go to version 1 of the general conference by using the name ``oldgen.''
Lines may be commented out by starting them with a # sign.
!<hl01>
%general2
gen_eral2:%general2
general1:%general1
oldgen_eral:%general1
tes_t1:%test1
mar_tian1:%martian1
mars:%martian1
bar_soom:%martian1
When Picospan or command-line Yapp are given a command to list all conferences, they simply print the public.txt file. This is a file with no fixed format that lives in the conference directory.
Backtalk needs a more formally formatted list of conferences that it can read, parse apart, and display in different formats in different interfaces. So Backtalk defines a new file called "confmenu" which lives in the conference directory.
In places where Backtalk co-exists with Picospan or command-line Yapp, Backtalk can be configured to use the traditional "public.txt" file instead, but it requires that it be entered in a consistant format so that it can be parsed. However, what that format should be can be configured. You can write your own regular expressions to parse a fairly broad range of file formats. Two sets of regular expressions are pre-written, one for a Grex-style public.txt file, one for an M-Net-style public.txt file. Those will be described here.
Web versions of Yapp seem to have a different conference menu database. I haven't figured out what exactly this is.
:Terrestrial Conferences:Conferences primarily relating to the planet earth. general:General:Our conference for general discussions oldgeneral:Old General:Previous version of the general conference :Interplanetary Conferences:Sponsored by your friends from NASA mars:Martian:Talk about Mars here venus:Venusian:Talk about Venus here :Test Conferences: test:Test:A place to experiment
You don't need to have any headers in the conference menu.
The first field in each line is the internal name of the conference. (This is blank for header lines). This must match a line in the conflist file. This is normally all lower case with no spaces.
The second field is the name of the conference to be displayed in the conference list. This can be mixed case and can have spaces, but should generally be something quite short. For headers it is the text of the header.
The third field is a longer description of the conference or group of conferences.
Lines starting with # signs are comments and are ignored.
@@@@@@@@@@@@@@@@@@@
@ OUR CONFERENCES @
@@@@@@@@@@@@@@@@@@@
Terrestrial Conferences:
general - Our conference for general discussions
oldgeneral - Previous version of the general conference
Interplanetary Conferences:
mars - Talk about Mars here
venus - Talk about Venus here
Test Conferences:
test - A place to experiment
(last modified 12/23/02)
Header lines start in column 0 (no leading spaces) and have a colon in them. The header text is from the start of the line through the colon. You cannot give descriptions.
Conference lines start with exactly one space. After the space is the internal conference name, followed by white spaces and dashes. After this is the description of the conference. The description must all fit on one line. The title of the conference will simply be a capitalized version of the conference name.
You may JOIN any of the following conferences: ********Terrestrial Conferences********* general......Our conference for general discussions oldgeneral...Previous version of the general conference *******Interplanetary Conferences******** mars.........Talk about Mars here venus........Talk about Venus here ***********Test Conferences************* test.........A place to experiment
All you have to do is provide two regular expressions - one that matches header lines, and one that matches conference lines. They should contain parenthesis that pick out the interesting substrings of the lines - the header text for headers, and the conference name and description for conferences.
This means that (1) headers and conferences must be described in one line, and (2) the conference name must appear before the conference description in conference lines. If this is true for your public.txt file, then Backtalk can probably use it.
!<pc02>
.general.cf
0
bob,roger
0
General
Later versions of Yapp allow sequences of keywords to be used instead of numeric modes. Backtalk also recognizes these. They are given at right in the table above. Picospan does not recognize these keywords.* Note that read-only conferences are not recognized by Picospan, though they are recognized by Yapp. If you are using Backtalk with Picospan, modes 20, 21, and 22 cannot be used.
0 all users may read and post [default] public 4 only users in the ulist or glist may read and post ulist 5 only users who know the password may read or post password 6 only users who are in the ulist or glist and know the password may read or post paranoid 8 all users may read and post (but item files not directly readable) protected 20* only users in the ulist or glsit may post / all may read ulist readonly 21* only users who know the password may post / all may read password readonly 22* only users who are in the ulist or glist and know the password may post / all may read paranoid readonly
If Backtalk was compiled to be Yapp-compatible mode, then the modes given in the config file will be ignored if an acl file exists. In non-Yapp compatible installations, the acl file will only be checked if the mode in the config file is 'acl' or a negative number.
Normally Backtalk will only pay attention to "acl" files only if the conference mode specified in the conference "config" file is negative or "acl". However, if compiled in Yapp-compatible mode, Backtalk will always check for the "acl" file and will ignore any conference modes set in the "config" file if it is found.
r +all
w +registered -f:observer
c +f:ulist
a +fwlist
The lines may appear in any order. Blank lines are accepted. Currently, the 'a' lines are ignored by Backtalk, since it doesn't actually include any tools to edit acl files.
r Who may join and read the conference. w Who may respond to existing items in the conference. c Who may create new topics in the conference. a Who may edit the acl file. # Comment.
After these characters will appear a sequence of keywords describing the tests that a user must pass to have that access. All tests on the line must be passed by a user to have the given access. If the keyword is prefixed with a '+' or has no prefix, then the user must pass the test. If the keyword is prefixed with a '-' then the user must fail the test.
Recognized tests are:
The '+all' keyword is actually a no-op, but the '-all' keyword is useful for banning anyone from doing something. Backtalk does not (yet) support '+all' for item entry or responding. You must always be 'registered' to perform these actions.
all All users. registered Users who have logged in. fwlist Fairwitnesses. originlist Users from sites listed in the 'originlist' file in the conference directory. Ignored by Backtalk. password Users who can give the conference password in the 'secret' file in the conference directory. f:filename Users whose logins are included in the named file in the conference directory. u=userlist Users whose logins appear in the comma-separated list. Not Yapp compatible. g:filename Users who are in at least one of group names listed in the named file in the conference directory. g=grouplist Users who are in at least one of comma-separated group names listed. Not Yapp compatible. sysop System administrators. Equivalent to 'g=cfadm' # Comment.
All the keywords except 'all' and 'secret' imply 'registered'. You have to be logged in to have a chance of passing them.
In Backtalk (though probably not in Yapp) 'fwlist' implies 'sysop', since system administrators are considered to be fairwitnesses in all conferences. If you wanted to permit something to fairwitnesses, but not sysops, you'd have to say '+fwlist -sysop'.
The 'f:' keyword is most commonly used as '+f:ulist' to say that a user must be in the ulist file, or as '-f:observer' to say that a user may not be in the observer file. Similarly, '+g:glist' says the user must be a group in the 'glist' file. However these may be used with any filename. Filenames starting with '/' are treated as full paths. Otherwise they are sought in the current conference's directory.
Because of this, it is one of the few binary files used by Backtalk. It must be rapidly accessible, and it does not have to be easily read or editted by humans. If a file is corrupted, you don't edit it, you delete it. The Backtalk distribution includes two programs, sumcheck and dumpsum that can be mildly useful in reading and repairing sum files, in the unlikely event that you should wish to do so. See man pages in the doc directory for more information.
The sum file contains a 24-byte header record, followed by a sequence of 16-byte item records. The sum file entry for item number N is always stored at offset 24 + 16(N-1).
Given a participation file name, we compute Checksum by doing
8 bytes char Magic Number Always "! \n". Identifies file type and version. 4 bytes int Checksum A checksum on the name of the participation file given in the config file. 4 bytes int Unknown Picospan always sets this to 0x00001537. I don't know what it means. 8 bytes long Sequence The value 0x12345678. Used for byte order checking.
set Checksum = 0;
for each character Char in the name
Checksum = (Checksum * 4) XOR Char
end
Backtalk puts this checksum into sum files it creates, but does
not check them when it reads an existing sum file.
Sum files are written using the native byte order of the computer that generated them. So if you have conferences on a shared file system, it is possible that different sum files may be created with different byte orders. We check the Sequence field to determine the byte order of the participation file. If it is not the native byte order, then the program must bytes when reading or writing.
In Picospan, ordinary items may have flag values 0x0030 or 0x003C. I don't know what makes the difference. In Backtalk and Yapp ordinary items flagged 0x0030. Additional bits are set for retired, frozen, synchronous or linked items. Entries for deleted items have the flags set to zero.
4 bytes unsigned int Flags Item flag bits. A boolean OR of the following:
0x0002 Item is retired. 0x000C Unknown. 0x0030 Item is active. 0x0040 Item is frozen. 0x0080 Item is synchronous. 0x0100 Item is linked. 4 bytes unsigned int Response Count The number of responses on the item, including the item text. 4 bytes unsigned int Response Date Date/time of last response to the item. 4 bytes unsigned int Item Date Date/time item was entered.
The times are standard Unix time integers (seconds since Jan 1, 1970). The response date should always match the 'mtime' value for the item file - the time at which the file was last modified.
The system is less picky about the item date, and it is set to slightly different values in different circumstances. When a new item is entered, it is set to the value the file's mtime had after the item was created. If an item is linked in from another conference, it is set to the time that the link was made. If the sum file ever has to be reconstructed, then it is set to match the time stamp on response zero (the item text). Picospan and Backtalk are not entirely consistant in the way they set this value, but I don't think it matters.
For the most part, Backtalk uses the traditional item file format used by Picospan and Yapp. The main extension is in how HTML and plain-text versions of responses are stored in installations that must be Picospan or Yapp compatible. This is done slightly different for Picospan and Yapp, and will be discussed at the end of this section.
!<ps02>
,HOur First Test Item
,R0000
,U232,janc
,AJan Wolter
,D3d7a9a89
,T
This is an item entered to test Backtalk.
This is the item text for that item.
It is a very good item.
,E
,R0000
,U123,joe
,AJoseph Cantata
,D3dbdfad5
,T
This is the first response to the very dull item
that was entered by Jan Wolter. This response too
is very dull.
,E
,R0000
,U232,janc
,AJan Wolter
,D3dbdfd2f
,T
How very dull! It is very good that this item is
so very dull.
,E
The second line is normally the item title, identified by a ",H" prefix.
The rest of the item file consists of a sequence of responses. Note that, the item text is simply the first response.
Each response consists of a ",R line that contains various flags for the response, a ",U" line that contains the login name and UID number of the author, a ",A" line that contains the author's full name, a ",D" line that contains the date of the response in hexadecimal, and then the text of the response, enclosed by ",T" and ",E" lines.
Any lines in the text that start with a comma are stored with two initial commas in a Picospan item file, or three initial commas in a Yapp item file. Backtalk follows Picospan in this, unless configured for Yapp compatibility.
Response flags are as follows:
,R0000 Regular response ,R0001 Hidden response ,R0003 Scribbled response
For some reason, it is common for the ",E" tag to be missing. Programs reading item files should tolerate this.
Yapp supports three additional directives. The "parent" link, ",P" usually appears between the ",D" and the ",T" lines:
,R0000
,U771,kate
,AKate Middleton
,D3b977d6a
,P2
,T
This is a response to response number 2.
,E
This causes the response to be displayed with a
"<response to #2>" message beside it.
There are also ",N" and ",M" lines that can appear
immediately after the ",T" line, but before the message text.
This are used when importing Usenet news messages and give the original
article number and message ID respectively. Backtalk understands the
",P" line but no the other two.
For editted responses, Backtalk stores an edit date in addition to the post date. This is a second hexadecimal string on the ",D" line.
,D3b977d6a 3b978ac1
The item modes are mostly stored in the item files permission bits. Linked items are detected by checking if the item's link count is greater than one. Frozen items have their owner-write permissions turned off. Retired items have their owner-execute permissions turned on. I forget how synchronous items are flagged - group execute, I think.
When Backtalk is configured for compatibility with Yapp or Picospan, then a plain text version of any HTML response is posted in addition to the HTML response. The plain text response is placed so that Yapp or Picospan will find that, while the HTML text is stashed someplace slighly unusual where Yapp or Picospan will ignore it.
For Picospan, this works like this:
,R0010
,U232,janc
,AJan Wolter
,D3b977d6a
I'm feeling very <STRONG>strong</STRONG>
and <EM>emphatic</EM> today.
,T
I'm feeling very strong
and emphatic today.
,E
Here the HTML response is inserted immediately before the ",T" line.
For Yapp, we additionally prefix each line of the HTML version of the response with ",:" characters:
,R0010
,U232,janc
,AJan Wolter
,D3b977d6a
,:I'm feeling very <STRONG>strong</STRONG>
,:and <EM>emphatic</EM> today.
,T
I'm feeling very strong
and emphatic today.
,E
Responses that have attachments include attachment lines, which are simply a ",X" prefix followed by an attachment handle. These appear after the ",A" line and before the ",D" line. More than one my appear. For example:
,R0010
,U232,janc
,AJan Wolter
,Xtest-1-8/doug1.gif
,Xtest-1-8/doug2.jpg
,D3b977d6a
,T
Here are some picture of my uncle doug.
,E
Index files reside in a subdirectory of the conference directory called "indexdir". The index file's name is the item number prefixed with an "@" character.
When Backtalk links an item, the index files will also be linked. If a link is created by Picospan or Yapp, no link is created and the items will likely end up with separate index files.
The format of an index file is simple. It is a sequence of 4-byte integers. The first is the offset of response 0 (the item text), the next is the offset of response 1, and so on. The offsets always point to the start of the ",R" field of the response. So to find response 412 in an item, we'd seek to offset 412*4 in the index file, read the long integer there, and then seek that offset in the item file.
In most Unix-login configurations, Backtalk changes the group of the files to one it can read/write. In Backtalk-login and Yapp 3.0 compatible Unix-login configurations, the files are owned by the cfadm account.
!<pr03>
Jan Wolter
1 76 3D1C5899
2 8 3D1C5A80
3 3 3D1C5A8A
4 20 3D1C5B69
5 18 3D1C5B80
6 58 3D1C5BA3
7 52 3D1C5CD8
8 -9 3D1CEF3F
9 -30 3D1CEF6A
The second line gives the user's alias in this conference.
The remaining lines describe items that the user has read. Each contains three columns.
The number of the last response read may be prefixed with a minus sign. This means that the item has been forgotten. Picospan recognizes "-0" as a distinct value from "0" - the former is an item that has never been read, but has been forgotten. The later is an un-read, un-forgotten item. Yapp does not understand "-0" and does not allow unread items to be forgotten.
The time of last reading is represented as a standard unix time value (the number of seconds since the begining of 1970), written in hexidecimal notation.
The time of last reading is used to distinguish between new and unseen items. An item has unseen responses if the number of responses in the item exceeds the last response read. An item has new responses if it is unseen AND the date of the last response in the item exceeds the date on which we last read responses.
These files are named the same as a Picospan file would have been, and are stored in the same locations, except that Backtalk accounts don't usually have .cfdir directories.
!<pr10>
alias=(Jan Wolter)
pistacho.background=(C0FFE0)
1
r=(76)
d=1025267865
2
r=(8)
d=1025267352
3
r=(3)
d=1025268362
4
r=(20)
d=1025268585
fav
5
r=(4-9,11,18)
d=1025268608
6
r=(58)
d=1025268643
7
r=(52)
d=1025268952
fav
8
r=(9)
d=1025306431
forgot
9
r=(30)
d=1025306474
forgot
The tag syntax is actually slightly different than regular tag files, for no good reason. A tag without a value (like "fav") is equivalent to setting it to integer 1 (that is, "fav=1"). If a string contains newlines, actual newlines are put in instead of "\n" values. Oh well.
In the conference-wide section, the 'alias' tag defines the user's name in the conferences.
In the per-item sections, the 'r' field is a selector describing all unread responses in the item. There is an implied '-$' at the end of each, so 'r=(9)' means we have not read responses 9-$. Currently these are always single digits, because, like Picospan, if we have read response 109, then we assume that we have read all that went before. But to support tree-style items, we need to remember exactly the set of responses which are still unread, so these will become more general selectors. If this is a single number (as it usually is), it can be written out as a number instead of a string, without the parentheses.
The d field is the date the item was last read. This is exactly like the date field in the traditional picospan participation file, except that it is written in decimal instead of hexadecimal.
Forgotten items are marked by a 'forgot' tag.
Extensions to the standard Picospan format include the 'fav' tag which is included if the item is one of the user's favorites, and the title flag which gives the user's personal title for the item.
So we maintain our extra information in an extra file, called the participation extension file. It's name is the same as the participation file, but with an 'x' appended to the name.
!<px01>
pistacho.background=(C0FFE0)
4
fav
5
r=(4-9,11,18)
7
fav