Help is invited and greatly appreciated. However, if you plan to do a significant modification that you would like me to incorporate into the base distribution, please try to discuss it with me first so we can coordinate the eventual integration.
If you would like to fund a particular development effort, either one I have described, or any other that you might have in mind, please do contact me.
We are starting to see variations on the HyperNews theme. Anyone can get the HyperNews source and start modifying it to give a different presentation for their forums and messages. The modified source can, in turn, be modified more by others, etc. This chaotic evolution with competition for readership should result in survival of the fittest. I will be happy to have contributed even if my initial HyperNews scheme ends up being mutated beyond recognition.
If HyperNews gets used a lot, the web of information and discussion will evolve along with the presentation of it. I expect there will be many entry points into the HyperNews web via different organizations of information and home pages, and no particular top node. Some parts of the web will be predominately information while other parts will have lots of discussion. Most discussion nodes should have at least some information associated with them. This is beginning to happen as evidenced by the growing number of HyperNews forums on other sites.
Even with archiving, it is occasionally appropriate to delete messages, move them to a different forum, or expire them after a certain age or event. HyperNews supports such modification of forums and messages via its administrative tools, although more work is needed to make it more convenient. Automatic reorganization is very difficult to do well, but it would be very valuable since manual reorganization is frequently too much work no matter how easy it is.
Automatic searching is also essential with any sufficiently large forum to avoid having to manually browse through the archives. Searching also serves to keep the forum smaller, ironically enough, by helping users find previous threads of interest rather than encouraging them to post new messages about the same old subjects. Searching can also be used to create virtual forums that are identified by the keywords (or the query in general) used to find a collection of messages. In fact, searching could be the means by which forums may be automatically reorganized into a set of virtual forums that are of long-term interest to the community as well as the host.
Sites such as DejaNews and Reference.com provide large-scale archiving of many usenet groups and mailing lists, and many other sites archive particular groups and lists as well as providing a web-based posting mechanism. As the web continues to grow along with the number of users, I believe these centralized message archive services will need become replicated and decentralized as usenet is now. But they also will need to become specialized to various communities as email lists and web-based forums are now.
icons next to messages that have been recently created. But the
ordering of the messages is the same, so a new message added to an old
thread might get ignored. Also, the display of the new icon is the same
for all users whether or not they have already seen each message.
This single order, static presentation has the advantage of preserving
the cachability of the pages, but it is not particularly useful for
on-line reading of new messages.
To help find messages that are new or changed, an alternative presentation could be generated by HyperNews using chronological order of creation or last-modified date. You, the reader, would still have to scroll through the list looking for the point at which you last read things, since each person comes into the system at a different time. This chronological ordering combined with chronological subdivision is also useful for finding older messages created during a certain date period.
A frequently requested feature, in addition to the chronological ordering, is to have HyperNews remember which messages each reader has read, or remember just the last date that messages were read within each forum, and only display the new (or changed) messages. Doing this per-reader customization on the server would clearly be unscalable for a large number of readers, but it would be OK if the number were sufficiently small. If we do this, we also need to support replication of HyperNews forums across many servers so readers could use a nearby server, similar to what is done with Usenet news servers for all newsgroups.
This per-reader memory of what the reader has read is starting to look an awful lot like news reader. Instead of making HyperNews into a usenet-like reader of web-base forums, how about use existing news readers to read HyperNews messages that are distributed via usenet? For more on that, see the discussion on the issue of HyperNews Usenet Gateways.
Another alternative is to deliver new or changed messages to subscribers by email, as HyperNews currently supports with its two-way email gateway. Notification via a mailing list for each forum seems like it is moving backwards, but email turns out to be appropriate technology in this case. Email delivery via a single mail server would be just as unscalable as the web-based per-reader customization except for a couple of facts. First, the number of subscribers is usually much smaller than the number of casual readers. Second, if web-based readers repeatedly poll HyperNews servers for new messages when there are usually not new messages, this is worse for the server than simply delivering new messages when they actually are posted. Finally, each new message can be delivered in the same way to all subscribers, thus enabling scalable delivery by grouping addresses near each other. In fact, this group delivery mode is one step along the path toward the ideal, discussed in A Protocol for Scalable Group and Public Annotations presented at the Spring 1995 WWW conference.
Here are a few of the most important modules and systems I am considering using, in no particular order. Some are already being used.
A note about Java. We have looked into developing a HyperNews-like system in Java (on the server side) and there are some significant advantages, but one important reason not to do so is, ironically, lack of portability. It is easier to get a Perl program running under CGI on an arbitrary server than installing a Java program with the Java runtime environment. I am sure that will change at some point. The same story is true of many other possible languages, but not all languages will be equally successful.