Initial revision

[helm.git] / helm / DEVEL / pxp / pxp / doc / README.xml

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE readme SYSTEM "readme.dtd" [

<!--
<!ENTITY url.ocaml           "http://caml.inria.fr/">
<!ENTITY url.xml-spec        "http://www.w3.org/TR/1998/REC-xml-19980210.html">
<!ENTITY url.jclark-xmltdata "ftp://ftp.jclark.com/pub/xml/xmltest.zip">
<!ENTITY url.gps-ocaml-download "http://people.darmstadt.netsurf.de/ocaml">
<!ENTITY url.markup-download    "&url.gps-ocaml-download;/markup-0.1.tar.gz">
<!ENTITY person.gps             '<a
  href="mailto:Gerd.Stolpmann@darmstadt.netsurf.de">Gerd Stolpmann</a>'>
-->

<!ENTITY % common SYSTEM "common.xml">
%common;

<!-- Special HTML config: -->
<!ENTITY % readme:html:up '<a href="../..">up</a>'>

<!ENTITY % config SYSTEM "config.xml">
%config;

]>

<readme title="README - PXP, the XML parser for O'Caml">
  <sect1>
    <title>Abstract</title>
    <p>
<em>PXP</em> is a validating parser for XML-1.0 which has been written
entirely in Objective Caml.
</p>

    <p>PXP is the new name of the parser formerly known as "Markup". 
PXP means "Polymorphic XML parser" and emphasizes its most useful
property: that the API is polymorphic and can be configured such that
different objects are used to store different types of elements.</p>
  </sect1>

  <sect1>
    <title>Download</title>
    <p>
You can download <em>PXP</em> as gzip'ed <a
href="&url.pxp-download;">tarball</a>. The parser needs the <a
href="&url.netstring-project;">Netstring</a> package (0.9.3). Note that PXP
requires O'Caml 3.00.
</p>
  </sect1>

  <sect1>
    <title>User's Manual</title>
    <p>
The manual is included in the distribution both as Postscript document and
bunch of HTML files. An online version can be found <a
                                                       href="&url.pxp-manual;">here</a>.
</p>
  </sect1>

  <sect1>
    <title>Author, Credits, Copying</title>
    <p>
<em>PXP</em> has been written by &person.gps;; it contains contributions by
Claudio Sacerdoti Coen. You may copy it as you like,
you may use it even for commercial purposes as long as the license conditions
are respected, see the file LICENSE coming with the distribution. It allows
almost everything. 
</p>

    <p>Thanks also to Alain Frisch and Haruo Hosoya for discussions and bug
reports.</p>
  </sect1>

  <sect1>
    <title>Description</title>
    <p>
<em>PXP</em> is a validating XML parser for <a
href="&url.ocaml;">O'Caml</a>. It strictly complies to the 
<a href="&url.xml-spec;">XML-1.0</a> standard.
</p>

    <p>The parser is simple to call, usually only one statement (function
call) is sufficient to parse an XML document and to represent it as object
tree.</p>

    <p>
Once the document is parsed, it can be accessed using a class interface.
The interface allows arbitrary access including transformations. One of
the features of the document representation is its polymorphic nature;
it is simple to add custom methods to the document classes. Furthermore,
the parser can be configured such that different XML elements are represented
by objects created from different classes. This is a very powerful feature,
because it simplifies the structure of programs processing XML documents.
</p>

    <p>
Note that the class interface does not comply to the DOM standard. It was not a
development goal to realize a standard API (industrial developers can this much
better than I); however, the API is powerful enough to be considered as
equivalent with DOM. More important, the interface is compatible with the
XML information model required by many XML-related standards.
</p>

    <sect2>
      <title>Detailed feature list</title>

      <ul>
        <li><p>The XML instance is validated against the DTD; any violation of
a validation constraint leads to the rejection of the instance. The validator
has been carefully implemented, and conforms strictly to the standard. If
needed, it is also possible to run the parser in a well-formedness mode.</p>
        </li>
        <li><p>If possible, the validator applies a deterministic finite
automaton to validate the content models. This ensures that validation can
always be performed in linear time. However, in the case that the content
models are not deterministic, the parser uses a backtracking algorithm which
can be much slower. - It is also possible to reject non-deterministic content
models.</p>
        </li>
        <li><p>In particular, the validator also checks the complicated rules
whether parentheses are properly nested with respect to entities, and whether
the standalone declaration is satisfied. On demand, it is checked whether the
IDREF attributes only refer to existing nodes.</p>
        </li>
        <li><p>Entity references are automatically resolved while the XML text
is being scanned. It is not possible to recognize in the object tree where a
referenced entity begins or ends; the object tree only represents the logical structure.</p>
        </li>
        <li><p>External entities are loaded using a configurable resolver
infrastructure. It is possible to connect the parser with an arbitrary XML source.</p>
        </li>
        <li><p>The parser can read XML text encoded in a variety of character
sets. Independent of this, it is possible to choose the encoding of the
internal representation of the tree nodes; the parser automatically converts
the input text to this encoding. Currently, the parser supports UTF-8 and
ISO-8859-1 as internal encodings.</p>
        </li>
        <li><p>The interface of the parser has been designed such that it is
best integrated into the language O'Caml. The first goal was simplicity of
usage which is achieved by many convenience methods and functions, and by
allowing the user to select which parts of the XML text are actually
represented in the tree. For example, it is possible to store processing
instructions as tree nodes, but the parser can also be configured such that
these instructions are put into hashtables. The information model is compatible
with the requirements of XML-related standards such as XPath.</p>
        </li>
        <li><p>In particular, the node tree can optionally contain or leave out
processing instructions and comments. It is also possible to generate a "super
root" object which is the parent of the root element. The attributes of
elements are normally not stored as nodes, but it is possible to get them
wrapped into nodes.</p>
        </li>
        <li><p>There is also an interface for DTDs; you can parse and access
sequences of declarations. The declarations are fully represented as recursive
O'Caml values.
</p>
        </li>
      </ul>
    </sect2>


    <sect2>
      <title>Code examples</title>
      <p>
This distribution contains several examples:</p>
      <ul>
        <li><p>
<em>validate:</em> simply parses a
document and prints all error messages
</p></li>

        <li><p>
<em>readme:</em> Defines a DTD for simple "README"-like documents, and offers
conversion to HTML and text files<footnote>This particular document is an
example of this DTD!</footnote>.
</p></li>

        <li><p>
<em>xmlforms:</em> This is already a
sophisticated application that uses XML as style sheet language and data
storage format. It shows how a Tk user interface can be configured by an
XML style, and how data records can be stored using XML.
</p></li>
      </ul>
    </sect2>

    <sect2>
      <title>Restrictions and missing features</title>
      <p>
The following restrictions apply that are not violations of the standard:
</p>
      <ul>
        <li><p>
The attributes "xml:space", and "xml:lang" are not supported specially.
  (The application can do this.)</p></li>

        <li><p>
The built-in support for SYSTEM and PUBLIC identifiers is limited to
  local file access. There is no support for catalogs. The parser offers
  a hook to add missing features.</p></li>

        <li><p>
It is currently not possible to check for interoperatibility with SGML.
</p></li>
      </ul>

<p>The following features are also missing:</p>
      <ul>
        <li><p>There is no special support for namespaces. (Perhaps in the next release?)</p>
        </li>
        <li><p>There is no support for XPATH or XSLT.</p>
        </li>
      </ul>
<p>However, I hope that these features will be implemented soon, either by
myself or by contributors (who are invited to do so).</p>
    </sect2>

    <sect2>
      <title>Recent Changes</title>
      <ul>
        <li>
          <p>Changed in 1.0:</p>
          <p>Support for document order.</p>
        </li>
        <li>
          <p>Changed in 0.99.8:</p>
          <p>Several fixes of bugs reported by Haruo Hosoya and Alain
Frisch.</p>
          <p>The class type "node" has been extended: you can go directly to
the next and previous nodes in the list; you can refer to nodes by
position.</p>
          <p>There are now some iterators for nodes: find, find_all,
find_element, find_all_elements, map_tree, iter_tree.</p>
          <p>Experimental support for viewing attributes as nodes; I hope that
helps Alain writing his XPath evaluator.</p>
          <p>The user's manual has been revised and is almost up to date.</p>
        </li>
        <li>
          <p>Changed in 0.99.7:</p>
          <p>There are now additional node types T_super_root, T_pinstr and
T_comment, and the parser is able to create the corresponding nodes.</p>
          <p>The functions for character set conversion have been moved to
the Netstring package; they are not specific for XML.</p>
        </li>
        <li>
          <p>Changed in 0.99.6:</p>
          <p>Implemented a check on deterministic content models. Added
an alternate validator basing on a DFA. - This means that now all mandatory
features for an XML-1.0 parser are implemented! The parser is now substantially
complete.</p>
        </li>
        <li>
          <p>Changed in 0.99.5:</p>
          <p>The handling of ID and IDREF attributes has changed. The
index of nodes containing an ID attribute is now separated from the document.
Optionally the parser now checks whether the IDREF attributes refer to
existing elements.</p>
          <p>The element nodes can optionally store the location in the
source XML code.</p>
          <p>The method 'write' writes the XML tree in every supported
encoding. (Successor of 'write_compact_as_latin1'.)</p>
          <p>Several smaller changes and fixes.</p>
        </li>
        <li>
          <p>Changed in 0.99.4:</p>
          <p>The module Pxp_reader has been modernized. The resolver classes
are simpler to use. There is now support for URLs.</p>
          <p>The interface of Pxp_yacc has been improved: The type 'source'
is now simpler. The type 'domspec' has gone; the new 'spec' is opaque and
performs better. There are some new parsing modes.</p>
          <p>Many smaller changes.</p>
        </li>
        <li>
          <p>Changed in 0.99.3:</p>
          <p>The markup_* modules have been renamed to pxp_*. There is a new
compatibility API that tries to be compatible with markup-0.2.10.</p>
          <p>The type "encoding" is now a polymorphic variant.</p>
        </li>
        <li>
          <p>Changed in 0.99.2:</p>
          <p>Added checks for the constraints about the standalone
declaration.</p>
          <p>Added regression tests about attribute normalization, 
attribute checks, standalone checks.</p>
          <p>Fixed some minor errors of the attribute normalization
function.</p>
          <p>The bytecode/native archives are now separated in
a general part, in a ISO-8859-1-relevant part, and a UTF-8-relevant
part. The parser can again be compiled with ocamlopt.</p>
        </li>
        <li>
          <p>Changed in 0.99.1:</p>
          <p>In general, this release is an early pre-release of the
next stable version 1.00. I do not recommend to use it for serious
work; it is still very experimental!</p>
          <p>The core of the parser has been rewritten using a self-written
parser generator.</p>
          <p>The lexer has been restructured, and can now handle UTF-8
encoded files.</p>
          <p>Numerous other changes.</p>
        </li>

<!--
        <li>
          <p>Changed in 0.2.10:</p>
          <p>Bugfix: in the "allow_undeclared_attributes" feature.</p>
          <p>Bugfix: in the methods write_compact_as_latin1.</p>
          <p>Improvement: The code produced by the codewriter module can be
faster compiled and with less memory usage.</p>
        </li>

        <li>
          <p>Changed in 0.2.9:</p>
          <p>New: The module Markup_codewriter generates for a given XML
tree O'Caml code that creates the same XML tree. This is useful for
applications which use large, constant XML trees.</p>
          <p>New: Documents and DTDs have a method write_compact_as_latin1
that writes an XML tree to a buffer or to a channel. (But it is not a pretty
printer...)</p>
          <p>Enhancement: If a DTD contains the processing instruction
<code>
&lt;?xml:allow_undeclared_attributes x?&gt;</code> 
where "x" is the name of an already declared element it is allowed that
instances of this element type have attributes that have not been declared.
</p>
          <p>New function Markup_types.string_of_exn that converts an
exception from Markup into a readable string.</p>
          <p>Change: The module Markup_reader contains all resolvers.
The resolver API is now stable.</p>
          <p>New parser modes processing_instructions_inline and
virtual_root that help locating processing instructions exactly (if needed).
</p>
          <p>Many bugs regarding CRLF handling have been fixed.</p>
          <p>The distributed tarball contains now the regression test suite.
</p>
          <p>The manual has been extended (but it is still incomplete and
still behind the code).</p>
        </li>
        <li>
          <p>Changed in 0.2.8:</p>
          <p>A bit more documentation (Markup_yacc).</p>
          <p>Bugfix: In previous versions, the second trial to refer to
an entity caused a Bad_character_stream exception. The reason was improper
re-initialization of the resolver object.</p>
        </li>
        <li>
          <p>Changed in 0.2.7:</p>
          <p>Added some methods in Markup_document.</p>
          <p>Bugfix: in method orphaned_clone</p>
        </li>
        <li>
          <p>Changed in 0.2.6:</p>
          <p>Enhancement: The config parameter has a new component
"errors_with_line_numbers". If "true", error exceptions come with line numbers 
(the default; and the only option in the previous versions); if "false"
the line numbers are left out (only character positions). The parser is 10 to
20 percent faster if the lines are not tracked.</p>
          <p>Enhancement: If a DTD contains the processing instruction
<code>
&lt;?xml:allow_undeclared_elements_and_notations?&gt;</code> 
it is allowed that
elements and notations are undeclared. However, the elements for which
declarations exist are still validated. The main effect is that the
keyword ALL in element declarations means that also undeclared elements
are permitted at this location.</p>
          <p>Bugfix in method "set_nodes" of class Markup_document.node_impl.
</p>
        </li>
        <li>
          <p>Changed in 0.2.5:</p>
          <p>If the XML source is a string (i.e. Latin1 some_string is passed
to the parser functions as source), resolving did not work properly in 
previous releases. This is now fixed.
</p>
        </li>
        <li>
          <p>Changed in 0.2.4:</p>
          <p>A problem with some kind of DTD that does not specify the name
of the root element was fixed. As a result, the "xmlforms" application works
again. Again thanks to Haruo.</p>
          <p>Due to the XML specs it is forbidden that parameter entities are
referenced within the internal subset if the referenced text is not a 
complete declaration itself. This is checked, but the check was too hard;
even in external entities referenced from the internal subset this rule
was enforced. This has been corrected; in external entities it is now possible
to use parameter entities in an unrestricted way.
</p>
        </li>
        <li>
          <p>Changed in 0.2.3:</p>
          <p>A fix for a problem when installing Markup on Solaris. 
Haruo detected the problem.</p>
        </li>
        <li>
          <p>Changed in 0.2.2:</p>
          <p>A single bugfix: The parser did not reject documents where the
root element was not the element declared as root element. Again thanks
to Claudio.</p>
        </li>
        <li>
          <p>Changed in 0.2.1:</p>
          <p>A single bugfix which reduces the number of warnings. Thanks
to Claudio for detecting the bug.</p>
        </li>
        <li>
          <p>Changed in 0.2:</p>
          <p>
Much more constraints are checked in the 0.2 release than in 0.1. Especially
that entities are properly nested is now guaranteed; parsed entities now always
match the corresponding production of the grammar.</p>
          <p>
Many weak checks have been turned into strong checks. For example, it is now
detected if the "version", "encoding", and "standalone" attributes of an XML
declaration are ordered in the right way.
</p>
          <p>
The error messages have been improved. 
</p>
        </li>
-->
      </ul>
    </sect2>
  </sect1>
</readme>