SUBDIRS = common
-XMLFILES=bookinfo.xml introduction.xml installation.xml \
- indexdata.xml \
- asn.xml tools.xml odr.xml comstack.xml server.xml license.xml \
- future.xml zoom.xml credits.xml gfs-options.xml \
- yaz.xml soap.xml gfs-virtual.xml gfs-synopsis.xml \
- std-oid-table.xml bib1-diag-table.xml srw-diag-table.xml manref.xml local.ent
+XMLFILES = book.xml \
+ gfs-options.xml gfs-virtual.xml gfs-synopsis.xml \
+ std-oid-table.xml bib1-diag-table.xml srw-diag-table.xml \
+ manref.xml local.ent
HTMLFILES = index.html
yaz-json-parse.1: yaz-json-parse-man.xml
$(MAN_COMPILE) $(srcdir)/yaz-json-parse-man.xml
-$(HTMLFILES): $(XMLFILES)
+$(HTMLFILES): $(XMLFILES) book.xml
rm -f *.html
- $(HTML_COMPILE) $(srcdir)/yaz.xml
+ $(HTML_COMPILE) $(srcdir)/book.xml
$(MANFILES): local.ent
-yaz.pdf: $(XMLFILES)
- $(PDF_COMPILE) $(srcdir)/yaz.xml
+yaz.pdf: $(XMLFILES) book.xml
+ $(PDF_COMPILE) $(srcdir)/book.xml && mv book.pdf yaz.pdf
yazj.pdf:
- jade -E14 -D $(srcdir) -d common/print.dsl -t tex $(srcdir)/common/xml.dcl $(srcdir)/yaz.xml
+ jade -E14 -D $(srcdir) -d common/print.dsl -t tex $(srcdir)/common/xml.dcl $(srcdir)/book.xml
rm -f yazj.pdf
- cp yaz.tex yazj.tex
+ cp book.tex yazj.tex
pdfjadetex yazj.tex
pdfjadetex yazj.tex >/dev/null
pdfjadetex yazj.tex >/dev/null
+++ /dev/null
- <chapter id="asn"><title>The Z39.50 ASN.1 Module</title>
- <sect1 id="asn.introduction"><title>Introduction</title>
- <para>
- The &asn; module provides you with a set of C struct definitions for the
- various PDUs of the Z39.50 protocol, as well as for the complex types
- appearing within the PDUs. For the primitive data types, the C
- representation often takes the form of an ordinary C language type,
- such as <literal>Odr_int</literal> which is equivalent to an integral
- C integer. For ASN.1 constructs that have no direct
- representation in C, such as general octet strings and bit strings,
- the &odr; module (see section <link linkend="odr">The ODR Module</link>)
- provides auxiliary definitions.
- </para>
- <para>
- The &asn; module is located in sub directory <filename>z39.50</filename>.
- There you'll find C files that implements encoders and decoders for the
- Z39.50 types. You'll also find the protocol definitions:
- <filename>z3950v3.asn</filename>, <filename>esupdate.asn</filename>,
- and others.
- </para>
- </sect1>
- <sect1 id="asn.preparing"><title>Preparing PDUs</title>
-
- <para>
- A structure representing a complex ASN.1 type doesn't in itself contain the
- members of that type. Instead, the structure contains
- <emphasis>pointers</emphasis> to the members of the type.
- This is necessary, in part, to allow a mechanism for specifying which
- of the optional structure (SEQUENCE) members are present, and which
- are not. It follows that you will need to somehow provide space for
- the individual members of the structure, and set the pointers to
- refer to the members.
- </para>
- <para>
- The conversion routines don't care how you allocate and maintain your
- C structures - they just follow the pointers that you provide.
- Depending on the complexity of your application, and your personal
- taste, there are at least three different approaches that you may take
- when you allocate the structures.
- </para>
-
- <para>
- You can use static or automatic local variables in the function that
- prepares the PDU. This is a simple approach, and it provides the most
- efficient form of memory management. While it works well for flat
- PDUs like the InitReqest, it will generally not be sufficient for say,
- the generation of an arbitrarily complex RPN query structure.
- </para>
- <para>
- You can individually create the structure and its members using the
- <function>malloc(2)</function> function. If you want to ensure that
- the data is freed when it is no longer needed, you will have to
- define a function that individually releases each member of a
- structure before freeing the structure itself.
- </para>
- <para>
- You can use the <function>odr_malloc()</function> function (see
- <xref linkend="odr.use"/> for details). When you use
- <function>odr_malloc()</function>, you can release all of the
- allocated data in a single operation, independent of any pointers and
- relations between the data. <function>odr_malloc()</function> is based on a
- "nibble-memory"
- scheme, in which large portions of memory are allocated, and then
- gradually handed out with each call to <function>odr_malloc()</function>.
- The next time you call <function>odr_reset()</function>, all of the
- memory allocated since the last call is recycled for future use (actually,
- it is placed on a free-list).
- </para>
- <para>
- You can combine all of the methods described here. This will often be
- the most practical approach. For instance, you might use
- <function>odr_malloc()</function> to allocate an entire structure and
- some of its elements, while you leave other elements pointing to global
- or per-session default variables.
- </para>
-
- <para>
- The &asn; module provides an important aid in creating new PDUs. For
- each of the PDU types (say, <function>Z_InitRequest</function>), a
- function is provided that allocates and initializes an instance of
- that PDU type for you. In the case of the InitRequest, the function is
- simply named <function>zget_InitRequest()</function>, and it sets up
- reasonable default value for all of the mandatory members. The optional
- members are generally initialized to null pointers. This last aspect
- is very important: it ensures that if the PDU definitions are
- extended after you finish your implementation (to accommodate
- new versions of the protocol, say), you won't get into trouble with
- uninitialized pointers in your structures. The functions use
- <function>odr_malloc()</function> to
- allocate the PDUs and its members, so you can free everything again with a
- single call to <function>odr_reset()</function>. We strongly recommend
- that you use the <literal>zget_*</literal>
- functions whenever you are preparing a PDU (in a C++ API, the
- <literal>zget_</literal>
- functions would probably be promoted to constructors for the
- individual types).
- </para>
- <para>
- The prototype for the individual PDU types generally look like this:
- </para>
- <synopsis>
- Z_<type> *zget_<type>(ODR o);
- </synopsis>
-
- <para>
- eg.:
- </para>
-
- <synopsis>
- Z_InitRequest *zget_InitRequest(ODR o);
- </synopsis>
-
- <para>
- The &odr; handle should generally be your encoding stream, but it
- needn't be.
- </para>
- <para>
- As well as the individual PDU functions, a function
- <function>zget_APDU()</function> is provided, which allocates
- a top-level Z-APDU of the type requested:
- </para>
-
- <synopsis>
- Z_APDU *zget_APDU(ODR o, int which);
- </synopsis>
-
- <para>
- The <varname>which</varname> parameter is (of course) the discriminator
- belonging to the <varname>Z_APDU</varname> <literal>CHOICE</literal> type.
- All of the interface described here is provided by the &asn; module, and
- you access it through the <filename>proto.h</filename> header file.
-
- </para>
- </sect1>
- <sect1 id="asn.external"><title>EXTERNAL Data</title>
-
- <para>
- In order to achieve extensibility and adaptability to different
- application domains, the new version of the protocol defines many
- structures outside of the main ASN.1 specification, referencing them
- through ASN.1 EXTERNAL constructs. To simplify the construction and
- access to the externally referenced data, the &asn; module defines a
- specialized version of the EXTERNAL construct, called
- <literal>Z_External</literal>.It is defined thus:
- </para>
-
- <screen>
-typedef struct Z_External
-{
- Odr_oid *direct_reference;
- int *indirect_reference;
- char *descriptor;
- enum
- {
- /* Generic types */
- Z_External_single = 0,
- Z_External_octet,
- Z_External_arbitrary,
-
- /* Specific types */
- Z_External_SUTRS,
- Z_External_explainRecord,
- Z_External_resourceReport1,
- Z_External_resourceReport2
-
- ...
-
- } which;
- union
- {
- /* Generic types */
- Odr_any *single_ASN1_type;
- Odr_oct *octet_aligned;
- Odr_bitmask *arbitrary;
-
- /* Specific types */
- Z_SUTRS *sutrs;
- Z_ExplainRecord *explainRecord;
- Z_ResourceReport1 *resourceReport1;
- Z_ResourceReport2 *resourceReport2;
-
- ...
-
- } u;
-} Z_External;
- </screen>
-
- <para>
- When decoding, the &asn; module will attempt to determine which
- syntax describes the data by looking at the reference fields
- (currently only the direct-reference). For ASN.1 structured data, you
- need only consult the <literal>which</literal> field to determine the
- type of data. You can the access the data directly through the union.
- When constructing data for encoding, you set the union pointer to point
- to the data, and set the <literal>which</literal> field accordingly.
- Remember also to set the direct (or indirect) reference to the correct
- OID for the data type.
- For non-ASN.1 data such as MARC records, use the
- <literal>octet_aligned</literal> arm of the union.
- </para>
-
- <para>
- Some servers return ASN.1 structured data values (eg. database
- records) as BER-encoded records placed in the
- <literal>octet-aligned</literal> branch of the EXTERNAL CHOICE.
- The ASN-module will <emphasis>not</emphasis> automatically decode
- these records. To help you decode the records in the application, the
- function
- </para>
-
- <screen>
- Z_ext_typeent *z_ext_gettypebyref(const oid *oid);
- </screen>
-
- <para>
- Can be used to retrieve information about the known, external data
- types. The function return a pointer to a static area, or NULL, if no
- match for the given direct reference is found. The
- <literal>Z_ext_typeent</literal>
- is defined as:
- </para>
-
- <screen>
-typedef struct Z_ext_typeent
-{
- int oid[OID_SIZE]; /* the direct-reference OID. */
- int what; /* discriminator value for the external CHOICE */
- Odr_fun fun; /* decoder function */
-} Z_ext_typeent;
- </screen>
-
- <para>
- The <literal>what</literal> member contains the
- <literal>Z_External</literal> union discriminator value for the
- given type: For the SUTRS record syntax, the value would be
- <literal>Z_External_sutrs</literal>.
- The <literal>fun</literal> member contains a pointer to the
- function which encodes/decodes the given type. Again, for the SUTRS
- record syntax, the value of <literal>fun</literal> would be
- <literal>z_SUTRS</literal> (a function pointer).
- </para>
-
- <para>
- If you receive an EXTERNAL which contains an octet-string value that
- you suspect of being an ASN.1-structured data value, you can use
- <literal>z_ext_gettypebyref</literal> to look for the provided
- direct-reference.
- If the return value is different from NULL, you can use the provided
- function to decode the BER string (see <xref linkend="odr.use"/>
- ).
- </para>
-
- <para>
- If you want to <emphasis>send</emphasis> EXTERNALs containing
- ASN.1-structured values in the occtet-aligned branch of the CHOICE, this
- is possible too. However, on the encoding phase, it requires a somewhat
- involved juggling around of the various buffers involved.
- </para>
- <para>
- If you need to add new, externally defined data types, you must update
- the struct above, in the source file <filename>prt-ext.h</filename>, as
- well as the encoder/decoder in the file <filename>prt-ext.c</filename>.
- When changing the latter, remember to update both the
- <literal>arm</literal> arrary and the list
- <literal>type_table</literal>, which drives the CHOICE biasing that
- is necessary to tell the different, structured types apart
- on decoding.
- </para>
-
- <note>
- <para>
- Eventually, the EXTERNAL processing will most likely
- automatically insert the correct OIDs or indirect-refs. First,
- however, we need to determine how application-context management
- (specifically the presentation-context-list) should fit into the
- various modules.
- </para>
- </note>
-
- </sect1>
- <sect1 id="asn.pdu"><title>PDU Contents Table</title>
-
- <para>
- We include, for reference, a listing of the fields of each top-level
- PDU, as well as their default settings.
- </para>
-
- <table frame="top" id="asn.default.initialize.request">
- <title>Default settings for PDU Initialize Request</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
- </entry></row>
- <row><entry>
- options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
- </entry></row>
- <row><entry>
- preferredMessageSize</entry><entry>Odr_int</entry><entry>30*1024
- </entry></row>
- <row><entry>
- maximumRecordSize</entry><entry>Odr_int</entry><entry>30*1024
- </entry></row>
- <row><entry>
- idAuthentication</entry><entry>Z_IdAuthentication</entry><entry>NULL
- </entry></row>
- <row><entry>
- implementationId</entry><entry>char*</entry><entry>"81"
- </entry></row>
- <row><entry>
- implementationName</entry><entry>char*</entry><entry>"YAZ"
- </entry></row>
- <row><entry>
- implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
- </entry></row>
- <row><entry>
- userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.initialize.response">
- <title>Default settings for PDU Initialize
- Response</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
- </entry></row>
- <row><entry>
- options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
- </entry></row>
- <row><entry>
- preferredMessageSize</entry><entry>Odr_int</entry><entry>30*1024
- </entry></row>
- <row><entry>
- maximumRecordSize</entry><entry>Odr_int</entry><entry>30*1024
- </entry></row>
- <row><entry>
- result</entry><entry>Odr_bool</entry><entry>TRUE
- </entry></row>
- <row><entry>
- implementationId</entry><entry>char*</entry><entry>"id)"
- </entry></row>
- <row><entry>
- implementationName</entry><entry>char*</entry><entry>"YAZ"
- </entry></row>
- <row><entry>
- implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
- </entry></row>
- <row><entry>
- userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.search.request">
- <title>Default settings for PDU Search Request</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- smallSetUpperBound</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- largeSetLowerBound</entry><entry>Odr_int</entry><entry>1
- </entry></row>
- <row><entry>
- mediumSetPresentNumber</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- replaceIndicator</entry><entry>Odr_bool</entry><entry>TRUE
- </entry></row>
- <row><entry>
- resultSetName</entry><entry>char *</entry><entry>"default"
- </entry></row>
- <row><entry>
- num_databaseNames</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- databaseNames</entry><entry>char **</entry><entry>NULL
- </entry></row>
- <row><entry>
- smallSetElementSetNames</entry><entry>Z_ElementSetNames
- </entry><entry>NULL
- </entry></row>
- <row><entry>
- mediumSetElementSetNames</entry><entry>Z_ElementSetNames
- </entry><entry>NULL
- </entry></row>
- <row><entry>
- preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
- </entry></row>
- <row><entry>
- query</entry><entry>Z_Query</entry><entry>NULL
- </entry></row>
- <row><entry>
- additionalSearchInfo</entry><entry>Z_OtherInformation
- </entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.search.response">
- <title>Default settings for PDU Search Response</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- resultCount</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- nextResultSetPosition</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- searchStatus</entry><entry>Odr_bool</entry><entry>TRUE
- </entry></row>
- <row><entry>
- resultSetStatus</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- presentStatus</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- records</entry><entry>Z_Records</entry><entry>NULL
- </entry></row>
- <row><entry>
- additionalSearchInfo</entry>
- <entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.present.request">
- <title>Default settings for PDU Present Request</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- resultSetId</entry><entry>char*</entry><entry>"default"
- </entry></row>
- <row><entry>
- resultSetStartPoint</entry><entry>Odr_int</entry><entry>1
- </entry></row>
- <row><entry>
- numberOfRecordsRequested</entry><entry>Odr_int</entry><entry>10
- </entry></row>
- <row><entry>
- num_ranges</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- additionalRanges</entry><entry>Z_Range</entry><entry>NULL
- </entry></row>
- <row><entry>
- recordComposition</entry><entry>Z_RecordComposition</entry><entry>NULL
- </entry></row>
- <row><entry>
- preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
- </entry></row>
- <row><entry>
- maxSegmentCount</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- maxRecordSize</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- maxSegmentSize</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.present.response">
- <title>Default settings for PDU Present Response</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- nextResultSetPosition</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- presentStatus</entry><entry>Odr_int</entry><entry>Z_PresentStatus_success
- </entry></row>
- <row><entry>
- records</entry><entry>Z_Records</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.delete.result.set.request">
- <title>Default settings for Delete Result Set Request
- </title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>referenceId
- </entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- deleteFunction</entry><entry>Odr_int</entry><entry>Z_DeleteResultSetRequest_list
- </entry></row>
- <row><entry>
- num_ids</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- resultSetList</entry><entry>char**</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.delete.result.set.response">
- <title>Default settings for Delete Result Set Response
- </title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- deleteOperationStatus</entry><entry>Odr_int</entry>
- <entry>Z_DeleteStatus_success</entry></row>
- <row><entry>
- num_statuses</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- deleteListStatuses</entry><entry>Z_ListStatus**</entry><entry>NULL
- </entry></row>
- <row><entry>
- numberNotDeleted</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- num_bulkStatuses</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- bulkStatuses</entry><entry>Z_ListStatus</entry><entry>NUL
- L</entry></row>
- <row><entry>
- deleteMessage</entry><entry>char*</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.scan.request">
- <title>Default settings for Scan Request
- </title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- num_databaseNames</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- databaseNames</entry><entry>char**</entry><entry>NULL
- </entry></row>
- <row><entry>
- attributeSet</entry><entry>Odr_oid</entry><entry>NULL
- </entry></row>
- <row><entry>
- termListAndStartPoint</entry><entry>Z_AttributesPlus...
- </entry><entry>NULL</entry></row>
- <row><entry>
- stepSize</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- numberOfTermsRequested</entry><entry>Odr_int</entry><entry>20
- </entry></row>
- <row><entry>
- preferredPositionInResponse</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.scan.response">
- <title>Default settings for Scan Response
- </title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- stepSize</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- scanStatus</entry><entry>Odr_int</entry><entry>Z_Scan_success
- </entry></row>
- <row><entry>
- numberOfEntriesReturned</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- positionOfTerm</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- entries</entry><entry>Z_ListEntris</entry><entry>NULL
- </entry></row>
- <row><entry>
- attributeSet</entry><entry>Odr_oid</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.trigger.resource.control.request">
- <title>Default settings for Trigger Resource Control Request </title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- requestedAction</entry><entry>Odr_int</entry><entry>
- Z_TriggerResourceCtrl_resou..
- </entry></row>
- <row><entry>
- prefResourceReportFormat</entry><entry>Odr_oid</entry><entry>NULL
- </entry></row>
- <row><entry>
- resultSetWanted</entry><entry>Odr_bool</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
-
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.resource.control.request">
- <title>Default settings for Resource Control Request</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- suspendedFlag</entry><entry>Odr_bool</entry><entry>NULL
- </entry></row>
- <row><entry>
- resourceReport</entry><entry>Z_External</entry><entry>NULL
- </entry></row>
- <row><entry>
- partialResultsAvailable</entry><entry>Odr_int</entry><entry>NULL
- </entry></row>
- <row><entry>
- responseRequired</entry><entry>Odr_bool</entry><entry>FALSE
- </entry></row>
- <row><entry>
- triggeredRequestFlag</entry><entry>Odr_bool</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.resource.control.response">
- <title>Default settings for Resource Control Response</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- continueFlag</entry><entry>bool_t</entry><entry>TRUE
- </entry></row>
- <row><entry>
- resultSetWanted</entry><entry>bool_t</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.access.control.request">
- <title>Default settings for Access Control Request</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- which</entry><entry>enum</entry><entry>Z_AccessRequest_simpleForm;
- </entry></row>
- <row><entry>
- u</entry><entry>union</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.access.control.response">
- <title>Default settings for Access Control Response</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- which</entry><entry>enum</entry><entry>Z_AccessResponse_simpleForm
- </entry></row>
- <row><entry>
- u</entry><entry>union</entry><entry>NULL
- </entry></row>
- <row><entry>
- diagnostic</entry><entry>Z_DiagRec</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.segment">
- <title>Default settings for Segment</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>value=0
- </entry></row>
- <row><entry>
- num_segmentRecords</entry><entry>Odr_int</entry><entry>0
- </entry></row>
- <row><entry>
- segmentRecords</entry><entry>Z_NamePlusRecord</entry><entry>NULL
- </entry></row>
- <row><entry>otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
- </tbody>
- </tgroup>
- </table>
-
- <table frame="top" id="asn.default.close">
- <title>Default settings for Close</title>
- <tgroup cols="3">
- <colspec colwidth="7*" colname="field"></colspec>
- <colspec colwidth="5*" colname="type"></colspec>
- <colspec colwidth="7*" colname="value"></colspec>
- <thead>
- <row>
- <entry>Field</entry>
- <entry>Type</entry>
- <entry>Default Value</entry>
- </row>
- </thead>
- <tbody>
-
- <row><entry>
- referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
- </entry></row>
- <row><entry>
- closeReason</entry><entry>Odr_int</entry><entry>Z_Close_finished
- </entry></row>
- <row><entry>
- diagnosticInformation</entry><entry>char*</entry><entry>NULL
- </entry></row>
- <row><entry>
- resourceReportFormat</entry><entry>Odr_oid</entry><entry>NULL
- </entry></row>
- <row><entry>
- resourceFormat</entry><entry>Z_External</entry><entry>NULL
- </entry></row>
- <row><entry>
- otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
- </entry></row>
-
- </tbody>
- </tgroup>
- </table>
-
- </sect1>
- </chapter>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
--- /dev/null
+<?xml version="1.0" standalone="no"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
+ "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
+[
+ <!ENTITY % local SYSTEM "local.ent">
+ %local;
+ <!ENTITY % entities SYSTEM "entities.ent">
+ %entities;
+ <!ENTITY % idcommon SYSTEM "common/common.ent">
+ %idcommon;
+]>
+<book>
+ <bookinfo>
+ <title>YAZ User's Guide and Reference</title>
+ <authorgroup>
+ <author><firstname>Sebastian</firstname><surname>Hammer</surname></author>
+ <author><firstname>Adam</firstname><surname>Dickmeiss</surname></author>
+ <author><firstname>Mike</firstname><surname>Taylor</surname></author>
+ <author><firstname>Heikki</firstname><surname>Levanto</surname></author>
+ <author><firstname>Dennis</firstname><surname>Schafroth</surname></author>
+ </authorgroup>
+ <releaseinfo>&version;</releaseinfo>
+ <copyright>
+ <year>©right-year;</year>
+ <holder>Index Data</holder>
+ </copyright>
+ <abstract>
+ <simpara>
+ This document is the programmer's guide and reference to the &yaz;
+ package version &version;. &yaz; is a compact toolkit that provides
+ access to the Z39.50 and SRU/Solr protocols, as well as a set of
+ higher-level tools for implementing the server and client
+ roles, respectively.
+ The documentation can be used on its own, or as a reference when
+ looking at the example applications provided with the package.
+ </simpara>
+ <simpara>
+ <inlinemediaobject>
+ <imageobject>
+ <imagedata fileref="common/id.png" format="PNG"/>
+ </imageobject>
+ <imageobject>
+ <imagedata fileref="common/id.eps" format="EPS"/>
+ </imageobject>
+ </inlinemediaobject>
+ </simpara></abstract>
+ </bookinfo>
+ <chapter id="introduction">
+ <title>Introduction</title>
+ <para>
+ &yaz; is a C/C++ library for information retrieval applications
+ using the Z39.50/SRU/Solr protocols for information retrieval.
+ </para>
+ <para>
+ Properties of &yaz;:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Complete
+ <ulink url="&url.z39.50;">Z39.50</ulink> version 3 support.
+ Amendments and Z39.50-2002 revision is supported.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Supports
+ <ulink url="&url.sru;">SRU GET/POST/SOAP</ulink>
+ version 1.1, 1.2 and 2.0 (over HTTP and HTTPS).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Includes BER encoders/decoders for the
+ <ulink url="&url.ill;">ISO ILL</ulink>
+ protocol.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Supports
+ <ulink url="&url.solr;">Solr</ulink> Web Service version 1.4.x
+ (client side only)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Supports the following transports: BER over TCP/IP
+ (<ulink url="&url.ber.over.tcpip;">RFC1729</ulink>),
+ BER over unix local socket, and
+ <ulink url="&url.http.1.1;">HTTP 1.1</ulink>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Secure Socket Layer support using
+ <ulink url="&url.gnutls;">GnuTLS</ulink>.
+ If enabled, &yaz; uses HTTPS transport (for SOAP) or
+ "Secure BER" (for Z39.50).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Offers
+ <ulink url="&url.zoom;">ZOOM</ulink> C API implementing
+ Z39.50, SRU and Solr Web Service.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The &yaz; library offers a set of useful utilities
+ related to the protocols, such as MARC (ISO2709) parser,
+ CCL (ISO8777) parser,
+ <ulink url="&url.cql;">CQL</ulink>
+ parser, memory management routines, character set conversion.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Portable code. &yaz; compiles out-of-the box on most Unixes and
+ on Windows using Microsoft Visual C++.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Fast operation. The C based BER encoders/decoders as well
+ as the server component of &yaz; is very fast.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Liberal license that allows for commercial use of &yaz;.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <sect1 id="introduction.reading">
+ <title>Reading this Manual</title>
+ <para>
+ Most implementors only need to read a fraction of the
+ material in thie manual, so a quick walkthrough of the chapters
+ is in order.
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ <xref linkend="installation"/> contains installation
+ instructions for &yaz;. You don't need reading this
+ if you expect to download &yaz; binaries.
+ However, the chapter contains information about how
+ to make <emphasis>your</emphasis> application link
+ with &yaz;.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="zoom"/> describes the ZOOM API of &yaz;.
+ This is definitely worth a read if you wish to develop a Z39.50/SRU
+ client.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="server"/> describes the generic frontend server
+ and explains how to develop server Z39.50/SRU applications for &yaz;.
+ Obviously worth reading if you're to develop a server.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="yaz-client"/> describes how to use the &yaz; Z39.50
+ client. If you're developer and wish to test your server
+ or a server from another party, you might find this chapter
+ useful.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="asn"/> documents the most commonly used Z39.50
+ C data structures offered by the &yaz; API. Client
+ developers using ZOOM and non-Z39.50 implementors may skip this.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="soap"/> describes how SRU and SOAP is used
+ in &yaz;. Only if you're developing SRU applications
+ this section is a must.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="tools"/> contains sections for the various
+ tools offered by &yaz;. Scan through the material quickly
+ and see what's relevant to you! SRU implementors
+ might find the <link linkend="cql">CQL</link> section
+ particularly useful.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="odr"/> goes through the details of the
+ ODR module which is the work horse that encodes and decodes
+ BER packages. Implementors using ZOOM only, do <emphasis>not</emphasis>
+ need reading this.
+ Most other Z39.50 implementors only need to read the first two
+ sections (<xref linkend="odr.introduction"/> and
+ <xref linkend="odr.use"/>).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <xref linkend="comstack"/> describes the network layer module
+ COMSTACK. Implementors using ZOOM or the generic frontend server
+ may skip this. Others, presumably, handling client/server
+ communication on their own should read this.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </sect1>
+ <sect1 id="introduction.api">
+ <title>The API</title>
+ <para>
+ The <ulink url="&url.yaz;">&yaz;</ulink>
+ toolkit offers several different levels of access to the
+ <ulink url="&url.z39.50;">ISO23950/Z39.50</ulink>,
+ <ulink url="&url.ill;">ILL</ulink> and
+ <ulink url="&url.sru;">SRU</ulink>
+ protocols.
+ The level that you need to use depends on your requirements, and
+ the role (server or client) that you want to implement.
+ If you're developing a client application you should consider the
+ <link linkend="zoom">ZOOM</link> API.
+ It is, by far, the easiest way to develop clients in C.
+ Server implementers should consider the
+ <link linkend="server">generic frontend server</link>.
+ None of those high-level APIs support the whole protocol, but
+ they do include most facilities used in existing Z39.50 applications.
+ </para>
+ <para>
+ If you're using 'exotic' functionality (meaning anything not included in
+ the high-level APIs), developing non-standard extensions to Z39.50 or
+ you're going to develop an ILL application you'll have to learn the lower
+ level APIs of &yaz;.
+ </para>
+ <para>
+ The YAZ toolkit modules is shown in figure <xref linkend="yaz.layer"/>.
+ </para>
+ <figure id="yaz.layer">
+ <title>YAZ layers</title>
+ <mediaobject>
+ <imageobject>
+ <imagedata fileref="apilayer.png" format="PNG"/>
+ </imageobject>
+ <imageobject>
+ <imagedata fileref="apilayer.eps" format="EPS"/>
+ </imageobject>
+ </mediaobject>
+ </figure>
+ <para>
+ There are four layers.
+ <itemizedlist>
+ <listitem>
+ <para>A client or server application (or both).
+ This layer includes ZOOM and the generic frontend server.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The second layer provides a C represenation of the
+ protocol units (packages) for Z39.50 ASN.1, ILL ASN.1,
+ SRU.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The third layer encodes and decodes protocol data units to
+ simple packages (buffer with certain length). The &odr; module
+ encodes and decodes BER whereas the HTTP modules encodes and
+ decodes HTTP ruquests/responses.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The lowest layer is &comstack; which exchanges the encoded packages
+ with a peer process over a network.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The &asn; module represents the ASN.1 definition of
+ the Z39.50 protocol. It establishes a set of type and
+ structure definitions, with one structure for each of the top-level
+ PDUs, and one structure or type for each of the contained ASN.1 types.
+ For primitive types, or other types that are defined by the ASN.1
+ standard itself (such as the EXTERNAL type), the C representation is
+ provided by the &odr; (Open Data Representation) subsystem.
+ </para>
+ <para>
+ &odr; is a basic mechanism for representing an
+ ASN.1 type in the C programming language, and for implementing BER
+ encoders and decoders for values of that type. The types defined in
+ the &asn; module generally have the prefix <literal>Z_</literal>, and
+ a suffix corresponding to the name of the type in the ASN.1
+ specification of the protocol (generally Z39.50-1995). In the case of
+ base types (those originating in the ASN.1 standard itself), the prefix
+ <literal>Odr_</literal> is sometimes seen. Either way, look for
+ the actual definition in either <filename>z-core.h</filename> (for the types
+ from the protocol), <filename>odr.h</filename> (for the primitive ASN.1
+ types).
+ The &asn; library also provides functions (which are, in turn,
+ defined using &odr; primitives) for encoding and decoding data values.
+ Their general form is
+ <funcsynopsis>
+ <funcprototype><funcdef>int <function>z_<replaceable>xxx</replaceable></function></funcdef>
+ <paramdef>ODR <parameter>o</parameter></paramdef>
+ <paramdef>Z_<replaceable>xxx</replaceable> **<parameter>p</parameter></paramdef>
+ <paramdef>int <parameter>optional</parameter></paramdef>
+ <paramdef>const char *<parameter>name</parameter></paramdef>
+ </funcprototype>
+ </funcsynopsis>
+ (note the lower-case "z" in the function name)
+ </para>
+ <note>
+ <para>
+ If you are using the premade definitions of the &asn; module, and you
+ are not adding new protocol of your own, the only parts of &odr; that you
+ need to worry about are documented in
+ <xref linkend="odr.use"/>.
+ </para>
+ </note>
+ <para>
+ When you have created a BER-encoded buffer, you can use the &comstack;
+ subsystem to transmit (or receive) data over the network. The &comstack;
+ module provides simple functions for establishing a connection
+ (passively or actively, depending on the role of your application),
+ and for exchanging BER-encoded PDUs over that connection. When you
+ create a connection endpoint, you need to specify what transport to
+ use (TCP/IP, SSL or UNIX sockets).
+ For the remainder of the connection's lifetime, you don't have
+ to worry about the underlying transport protocol at all - the &comstack;
+ will ensure that the correct mechanism is used.
+ </para>
+ <para>
+ We call the combined interfaces to &odr;, &asn;, and &comstack; the service
+ level API. It's the API that most closely models the Z39.50
+ service/protocol definition, and it provides unlimited access to all
+ fields and facilities of the protocol definitions.
+ </para>
+ <para>
+ The reason that the &yaz; service-level API is a conglomerate of the
+ APIs from three different submodules is twofold. First, we wanted to allow
+ the user a choice of different options for each major task. For instance,
+ if you don't like the protocol API provided by &odr;/&asn;, you
+ can use SNACC or BERUtils instead, and still have the benefits of the
+ transparent transport approach of the &comstack; module. Secondly,
+ we realize that you may have to fit the toolkit into an existing
+ event-processing structure, in a way that is incompatible with
+ the &comstack; interface or some other part of &yaz;.
+ </para>
+ </sect1>
+ </chapter>
+ <chapter id="installation">
+ <title>Compilation and Installation</title>
+ <sect1 id="installation-introduction">
+ <title>Introduction</title>
+ <para>
+ The latest version of the software will generally be found at:
+ </para>
+ <para>
+ <ulink url="&url.yaz.download;"/>
+ </para>
+ <para>
+ We have tried our best to keep the software portable, and on many
+ platforms, you should be able to compile everything with little or
+ no changes.
+ </para>
+ <para>
+ The software is regularly tested on
+ <ulink url="&url.debian;">Debian GNU/Linux</ulink>,
+ <ulink url="&url.centos;">CentOS</ulink>,
+ <ulink url="&url.ubuntu;">Ubuntu Linux</ulink>,
+ <ulink url="&url.freebsd;">FreeBSD (i386)</ulink>,
+ <ulink url="&url.macosx;">MAC OSX</ulink>,
+ <ulink url="&url.solaris;">Solaris</ulink>,
+ Windows 7, Windows XP.
+ </para>
+ <para>
+ Some versions have be known to work on HP/UX,
+ DEC Unix, <ulink url="&url.netbsd;">NetBSD</ulink>,
+ <ulink url="&url.openbsd;">OpenBSD</ulink>,
+ IBM AIX,
+ Data General DG/UX (with some CFLAGS tinkering),
+ SGI/IRIX, DDE Supermax, Apple Macintosh (using the Codewarrior programming
+ environment and the GUSI socket libraries),
+ IBM AS/400 .
+ </para>
+ <para>
+ If you move the software to other platforms, we'd be grateful if you'd
+ let us know about it. If you run into difficulties, we will try to help
+ if we can, and if you solve the problems, we would be happy to include
+ your fixes in the next release. So far, we have mostly avoided
+ <literal>#ifdefs</literal> for individual platforms, and we'd
+ like to keep it that way as far as it makes sense.
+ </para>
+ <para>
+ We maintain a mailing-list for the purpose of announcing new releases and
+ bug-fixes, as well as general discussion. Subscribe by
+ filling-in the form
+ <ulink url="&url.yaz.mailinglist;">here</ulink>.
+ General questions and problems can be directed at
+ <ulink url="&url.yaz.mail;"/>, or the address given at the top of
+ this document.
+ </para>
+ </sect1>
+ <sect1 id="installation.unix"><title>UNIX</title>
+ <para>
+ We provide
+ <ulink url="&url.debian;">Debian GNU/Linux</ulink> (i386 and amd64),
+ <ulink url="&url.ubuntu;">Ubuntu</ulink> (i386 and amd64)
+ and
+ <ulink url="&url.centos;">CentOS</ulink> (amd64 only) packages for &yaz;.
+ You should be able to create packages for other CPUs by building
+ them from the source package.
+ </para>
+ <para>
+ YAZ is also part of several packages repositories. Some of them are
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Solaris CSW: <ulink url="http://www.opencsw.org/packages/yaz/"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Solaris: <ulink url="http://unixpackages.com"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ FreeBSD: <ulink url="http://www.freshports.org/net/yaz"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Debian: <ulink url="http://packages.debian.org/search?keywords=yaz"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Ubuntu: <ulink url="https://launchpad.net/ubuntu/+source/yaz"/>
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ NetBSD:
+ <ulink url="http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/net/yaz/README.html"/>
+ </para>
+ </listitem>
+ </itemizedlist>
+ <sect2 id="installation.source.unix">
+ <title>Compiling from source on Unix</title>
+ <para>
+ Note that if your system doesn't have a native ANSI C compiler, you may
+ have to acquire one separately. We recommend
+ <ulink url="&url.gcc;">GCC</ulink>.
+ </para>
+ <para>
+ If you wish to use character set conversion facilities in &yaz; or if you
+ are compiling &yaz; for use with Zebra it is a good idea to ensure that
+ the iconv library is installed. Some Unixes today already have it
+ - if not, we suggest
+ <ulink url="&url.libiconv;">GNU libiconv</ulink>.
+ </para>
+ <para>
+ YAZ 3.0.16 and later includes a wrapper for the
+ <ulink url="&url.icu;">ICU</ulink>
+ (International Components for Unicode).
+ In order to use this, the developer version of the ICU library
+ must be available. ICU support is recommended for applications
+ such as Pazpar2 and Zebra.
+ </para>
+ <para>
+ The <ulink url="&url.libxslt;">libxslt</ulink>,
+ <ulink url="&url.libxml2;">libxml2</ulink> librararies are required
+ if &yaz; is to support SRU/Solr.
+ These libraries are very portable and should compile out-of-the
+ box on virtually all Unix platforms. It is available in binary
+ forms for Linux and others.
+ </para>
+ <para>
+ The GNU tools
+ <ulink url="&url.autoconf;">Autoconf</ulink>,
+ <ulink url="&url.automake;">Automake</ulink> and
+ <ulink url="&url.libtool;">Libtool</ulink>
+ are used to generate Makefiles and configure &yaz; for the system.
+ You do <emphasis>not</emphasis> these tools unless you're using the
+ Git version of &yaz;.
+ </para>
+ <para>
+ The CQL parser for &yaz; is built using
+ GNU <ulink url="&url.bison;">Bison</ulink>.
+ This tool is only needed if you're using the Git version of &yaz;.
+ </para>
+ <para>
+ &yaz; includes a tiny ASN.1 compiler. This compiler is
+ written in <ulink url="&url.tcl;">Tcl</ulink>.
+ But as for Bison you do not need it unless you're using Git
+ version of &yaz; or you're using the compiler to built own codecs
+ for private ASN.1.
+ </para>
+ <para>
+ Generally it should be sufficient to run configure without options,
+ like this:
+ </para>
+ <screen>
+ ./configure
+ </screen>
+ <para>
+ The configure script attempts to use use the C compiler specified by
+ the <literal>CC</literal> environment variable. If not set, GNU C will be
+ used if it is available. The <literal>CFLAGS</literal> environment
+ variable holds options to be passed to the C compiler. If you're using
+ Bourne-compatible shell you may pass something like this to use a
+ particular C compiler with optimization enabled:
+ </para>
+ <screen>
+ CC=/opt/ccs/bin/cc CFLAGS=-O ./configure
+ </screen>
+ <para>
+ To customize &yaz;, the configure script also accepts a set of options.
+ The most important are:
+ <variablelist>
+ <varlistentry>
+ <term>
+ <literal>--prefix</literal>=<replaceable>prefix</replaceable>
+ </term>
+ <listitem>
+ <para>Specifies installation prefix for &yaz;. This is
+ only needed if you run <literal>make install</literal> later to
+ perform a "system" installation. The prefix is
+ <literal>/usr/local</literal> if not specified.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--enable-tcpd</literal>
+ </term>
+ <listitem>
+ <para>The front end server will be built using Wietse's
+ <ulink url="&url.tcpwrapper;">TCP wrapper library</ulink>.
+ It allows you to allow/deny clients depending on IP number.
+ The TCP wrapper library is often used in GNU/Linux and
+ BSD distributions.
+ See
+ <citerefentry>
+ <refentrytitle>hosts_access</refentrytitle>
+ <manvolnum>5</manvolnum>
+ </citerefentry>
+ and
+ <citerefentry>
+ <refentrytitle>tcpd</refentrytitle>
+ <manvolnum>8</manvolnum>
+ </citerefentry>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--enable-threads</literal>
+ </term>
+ <listitem>
+ <para>&yaz; will be built using POSIX threads.
+ Specifically, <constant>_REENTRANT</constant> will be defined during
+ compilation.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--disable-shared</literal>
+ </term>
+ <listitem>
+ <para>The make process will not create shared
+ libraries (also known as shared objects <filename>.so</filename>).
+ By default, shared libraries are created -
+ equivalent to <literal>--enable-shared</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--disable-shared</literal>
+ </term>
+ <listitem>
+ <para>The make process will not create
+ static libraries (<filename>.a</filename>).
+ By default, static libraries are created -
+ equivalent to <literal>--enable-static</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-iconv</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>Compile &yaz; with iconv library in directory
+ <replaceable>prefix</replaceable>. By default configure will
+ search for iconv on the system. Use this option if it
+ doesn't find iconv. Alternatively,
+ <literal>--without-iconv</literal>, can be uset to force &yaz;
+ not to use iconv.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-xslt</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>Compile &yaz; with
+ <ulink url="&url.libxslt;">libxslt</ulink> in directory
+ <replaceable>prefix</replaceable>.
+ Use this option if you want XSLT and XML support.
+ By default, configure will
+ search for libxslt on the system. Use this option if it
+ libxslt is not found automatically. Alternatively,
+ <literal>--without-xslt</literal>, can be used to force &yaz;
+ not to use libxslt.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-xml2</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>Compile &yaz; with
+ <ulink url="&url.libxml2;">libxml2</ulink> in directory
+ <replaceable>prefix</replaceable>.
+ Use this option if you want &yaz; to use XML and support SRU/Solr.
+ By default, configure will
+ search for libxml2 on the system. Use this option if it
+ libxml2 is not found automatically. Alternatively,
+ <literal>--without-xml2</literal>, can be used to force &yaz;
+ not to use libxml2.
+ </para>
+ <para>
+ Note that option <literal>--with-xslt</literal>
+ also enables libxml2.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-gnutls</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>&yaz; will be linked with the GNU TLS libraries and
+ an SSL COMSTACK will be provided. By default configure enables
+ SSL support for YAZ if the GNU TLS development libraries are found
+ on the system.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-icu</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>&yaz; will be linked the
+ <ulink url="&url.icu;">ICU</ulink> library in the prefix if given.
+ If prefix is not given, the libraries exposed by the script
+ <application>icu-config</application> will be used if found.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <literal>--with-libgcrypt</literal>[=<replaceable>prefix</replaceable>]
+ </term>
+ <listitem>
+ <para>&yaz; will be linked with
+ <ulink url="&url.libgcrypt;">Libgcrypt</ulink> in the prefix if given.
+ If prefix is not given, the libraries exposed by the script
+ <application>libgcrypt-config</application> will be used if found.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>--with-memcached</literal>
+ </term>
+ <listitem>
+ <para>&yaz; will be linked with
+ <ulink url="&url.libmemcached;">libMemcached</ulink> to allow
+ for result-set caching for ZOOM.
+ The prefix can not be given. Note that YAZ will only search
+ for libMemcached if Libgcrypt is also enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ When configured, build the software by typing:
+ <screen>
+ make
+ </screen>
+ </para>
+ <para>
+ The following files are generated by the make process:
+ <variablelist>
+ <varlistentry>
+ <term><filename>src/libyaz.la</filename></term>
+ <listitem><para>
+ Main &yaz; library. This is no ordinary library. It's
+ a Libtool archive.
+ By default, &yaz; creates a static library in
+ <filename>lib/.libs/libyaz.a</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>src/libyaz_server.la</filename></term>
+ <listitem><para>
+ Generic Frontend server. This is an add-on for libyaz.la.
+ Code in this library uses POSIX threads functions - if POSIX
+ threads are available on the platform.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>src/libyaz_icu.la</filename></term>
+ <listitem><para>
+ Functions that wrap the ICU library.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>ztest/yaz-ztest</filename></term>
+ <listitem><para>Test Z39.50 server.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>client/yaz-client</filename></term>
+ <listitem><para>Z39.50 client for testing the protocol.
+ See chapter <link linkend="yaz-client">
+ YAZ client</link> for more information.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-config</filename></term>
+ <listitem><para>A Bourne-shell script, generated by configure, that
+ specifies how external applications should compile - and link with
+ &yaz;.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-asncomp</filename></term>
+ <listitem><para>The ASN.1 compiler for &yaz;. Requires the
+ Tcl Shell, <application>tclsh</application>, in
+ <literal>PATH</literal> to operate.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-iconv</filename></term>
+ <listitem><para>This program converts data in one character set to
+ another. This command exercises the YAZ character set
+ conversion API.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-marcdump</filename></term>
+ <listitem><para>This program parses ISO2709 encoded MARC records
+ and prints them in line-format or XML.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-icu</filename></term>
+ <listitem><para>This program exposes the ICU wrapper library if that
+ is enabled for YAZ. Only if ICU is available this program is
+ useful.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>util/yaz-url</filename></term>
+ <listitem><para>This program is a simple HTTP page fetcher ala
+ wget or curl.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>zoom/zoomsh</filename></term>
+ <listitem><para>
+ A simple shell implemented on top of the
+ <link linkend="zoom">ZOOM</link> functions.
+ The shell is a command line application that allows you to enter
+ simple commands to perform ZOOM operations.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>zoom/zoomtst1</filename>,
+ <filename>zoom/zoomtst2</filename>, ..</term>
+ <listitem><para>
+ Several small applications that demonstrates the ZOOM API.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ If you wish to install &yaz; in system directories
+ <filename>/usr/local/bin</filename>,
+ <filename>/usr/local/lib</filename> .. etc, you can type:
+ </para>
+ <screen>
+ make install
+ </screen>
+ <para>
+ You probably need to have root access in order to perform this.
+ You must specify the <literal>--prefix</literal> option for configure if
+ you wish to install &yaz; in other directories than the default
+ <filename>/usr/local/</filename>.
+ </para>
+ <para>
+ If you wish to perform an un-installation of &yaz;, use:
+ </para>
+ <screen>
+ make uninstall
+ </screen>
+ <para>
+ This will only work if you haven't reconfigured &yaz; (and therefore
+ changed installation prefix). Note that uninstall will not
+ remove directories created by make install, e.g.
+ <filename>/usr/local/include/yaz</filename>.
+ </para>
+ </sect2>
+ <sect2 id="installation-linking-yaz-unix">
+ <title>How to make apps using YAZ on UNIX</title>
+ <para>
+ This section describes how to compile - and link your own
+ applications using the &yaz; toolkit.
+ If you're used to Makefiles this shouldn't be hard. As for
+ other libraries you have used before, you have to set a proper include
+ path for your C/C++ compiler and specify the location of
+ &yaz; libraries. You can do it by hand, but generally we suggest
+ you use the <filename>yaz-config</filename> that is generated
+ by <filename>configure</filename>. This is especially
+ important if you're using the threaded version of &yaz; which
+ require you to pass more options to your linker/compiler.
+ </para>
+ <para>
+ The <filename>yaz-config</filename> script accepts command line
+ options that makes the <filename>yaz-config</filename> script print
+ options that you should use in your make process.
+ The most important ones are:
+ <literal>--cflags</literal>, <literal>--libs</literal>
+ which prints C compiler flags, and linker flags respectively.
+ </para>
+ <para>
+ A small and complete <literal>Makefile</literal> for a C
+ application consisting of one source file,
+ <filename>myprog.c</filename>, may look like this:
+ <screen>
+ YAZCONFIG=/usr/local/bin/yaz-config
+ CFLAGS=`$(YAZCONFIG) --cflags`
+ LIBS=`$(YAZCONFIG) --libs`
+ myprog: myprog.o
+ $(CC) $(CFLAGS) -o myprog myprog.o $(LIBS)
+ </screen>
+ </para>
+ <para>
+ The CFLAGS variable consists of a C compiler directive that will set
+ the include path to the <emphasis>parent</emphasis> directory
+ of <filename>yaz</filename>. That is, if &yaz; header files were
+ installed in <filename>/usr/local/include/yaz</filename>,
+ then include path is set to <filename>/usr/local/include</filename>.
+ Therefore, in your applications you should use
+ <screen>
+ #include <yaz/proto.h>
+ </screen>
+ and <emphasis>not</emphasis>
+ <screen>
+ #include <proto.h>
+ </screen>
+ </para>
+ <para>
+ For Libtool users, the <filename>yaz-config</filename> script provides
+ a different variant of option <literal>--libs</literal>, called
+ <literal>--lalibs</literal> that returns the name of the
+ Libtool archive(s) for &yaz; rather than the ordinary ones.
+ </para>
+ <para>
+ For applications using the threaded version of &yaz;,
+ specify <literal>threads</literal> after the
+ other options. When <literal>threads</literal> is given,
+ more flags and linker flags will be printed by
+ <filename>yaz-config</filename>. If our previous example was
+ using threads, you'd have to modify the lines that set
+ <literal>CFLAGS</literal> and <literal>LIBS</literal> as
+ follows:
+ <screen>
+ CFLAGS=`$(YAZCONFIG) --cflags threads`
+ LIBS=`$(YAZCONFIG) --libs threads`
+ </screen>
+ There is no need specify POSIX thread libraries in your Makefile.
+ The <literal>LIBS</literal> variable includes that as well.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="installation.win32">
+ <title>WIN32</title>
+ <para>The easiest way to install YAZ on Windows is by downloading
+ an installer from
+ <ulink url="&url.yaz.download.win32;">here</ulink>.
+ The installer comes with source too - in case you wish to
+ compile YAZ with different compiler options, etc.
+ </para>
+
+ <sect2 id="installation.win32.source">
+ <title>Compiling from Source on WIN32</title>
+ <para>
+ &yaz; is shipped with "makefiles" for the NMAKE tool that comes
+ with <ulink url="&url.vstudio;">
+ Microsoft Visual Studio</ulink>. It has been tested with
+ Microsoft Visual Studio 2003/2005/2008.
+ </para>
+ <para>
+ Start a command prompt and switch the sub directory
+ <filename>WIN</filename> where the file <filename>makefile</filename>
+ is located. Customize the installation by editing the
+ <filename>makefile</filename> file (for example by using notepad).
+ The following summarizes the most important settings in that file:
+ <variablelist>
+ <varlistentry>
+ <term><literal>DEBUG</literal></term>
+ <listitem><para>
+ If set to 1, the software is
+ compiled with debugging libraries (code generation is
+ multi-threaded debug DLL).
+ If set to 0, the software is compiled with release libraries
+ (code generation is multi-threaded DLL).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_TCL</literal>, <literal>TCL</literal></term>
+ <listitem><para>
+ If <literal>HAVE_TCL</literal> is set to 1, nmake will
+ use the ASN.1 compiler (<ulink url="&url.tcl;">Tcl</ulink> based).
+ You must set <literal>TCL</literal> to the full path of the Tcl
+ interpreter. A Windows version of Tcl is part of
+ <ulink url="&url.gitwindows;">Git for Windows</ulink>.
+ </para>
+ <para>
+ If you do not have Tcl installed, set
+ <literal>HAVE_TCL</literal> to 0.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_BISON</literal>,
+ <literal>BISON</literal></term>
+ <listitem><para>
+ If GNU Bison is present, you might set <literal>HAVE_BISON</literal>
+ to 1 and specify the Bison executable in <literal>BISON</literal>.
+ Bison is only required if you use the Git version of
+ YAZ or if you modify the grammar for CQL
+ (<filename>cql.y</filename>).
+ </para>
+ <para>
+ A Windows version of GNU Bison is part of
+ <ulink url="&url.gitwindows;">Git for Windows</ulink>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_ICONV</literal>,
+ <literal>ICONV_DIR</literal></term>
+ <listitem><para>
+ If <literal>HAVE_ICONV</literal> is set to 1, YAZ is compiled
+ with iconv support. In this configuration, set
+ <literal>ICONV_DIR</literal> to the iconv source directory.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_LIBXML2</literal>,
+ <literal>LIBXML2_DIR</literal></term>
+ <listitem>
+ <para>
+ If <literal>HAVE_LIBXML2</literal> is set to 1, YAZ is compiled
+ with SRU support. In this configuration, set
+ <literal>LIBXML2_DIR</literal> to the
+ <ulink url="&url.libxml2;">libxml2</ulink> source directory
+ and
+ <literal>ZLIB_DIR</literal> to the zlib directory.
+ </para>
+ <para>
+ Windows versions of libxslt, libxml2, zlib and iconv can be found
+ <ulink url="&url.libxml2.download.win32;">
+ Igor Zlatković' site</ulink>.
+ </para>
+ <note>
+ <para>
+ YAZ is not using zlib but libxml2 is depending on it.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_LIBXSLT</literal>,
+ <literal>LIBXSLT_DIR</literal></term>
+ <listitem>
+ <para>
+ If <literal>HAVE_LIBXSLT</literal> is set to 1, YAZ is compiled
+ with XSLT support. In this configuration, set
+ <literal>LIBXSLT_DIR</literal> to the
+ <ulink url="&url.libxslt;">libxslt</ulink> source directory.
+ </para>
+ <note>
+ <para>
+ libxslt depends libxml2.
+ </para>
+ </note>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>HAVE_ICU</literal>,
+ <literal>ICU_DIR</literal></term>
+ <listitem>
+ <para>
+ If <literal>HAVE_ICU</literal> is set to 1, YAZ is compiled
+ with <ulink url="&url.icu;">ICU</ulink> support.
+ In this configuration, set
+ <literal>ICU_DIR</literal> to the
+ <ulink url="&url.icu;">ICU</ulink> source directory.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ When satisfied with the settings in the makefile, type
+ <screen>
+ nmake
+ </screen>
+ </para>
+ <note>
+ <para>
+ If the <filename>nmake</filename> command is not found on your system
+ you probably haven't defined the environment variables required to
+ use that tool. To fix that, find and run the batch file
+ <filename>vcvars32.bat</filename>. You need to run it from within
+ the command prompt or set the environment variables "globally";
+ otherwise it doesn't work.
+ </para>
+ </note>
+ <para>
+ If you wish to recompile &yaz; - for example if you modify
+ settings in the <filename>makefile</filename> you can delete
+ object files, etc by running.
+ <screen>
+ nmake clean
+ </screen>
+ </para>
+ <para>
+ The following files are generated upon successful compilation:
+ <variablelist>
+ <varlistentry>
+ <term><filename>bin/yaz&soversion;.dll</filename> /
+ <filename>bin/yaz&soversion;d.dll</filename></term>
+ <listitem><para>
+ &yaz; Release/Debug DLL.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>lib/yaz&soversion;.lib</filename> /
+ <filename>lib/yaz&soversion;d.lib</filename></term>
+ <listitem><para>
+ Import library for <filename>yaz&soversion;.dll</filename> /
+ <filename>yaz&soversion;d.dll</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz_cond&soversion;.dll</filename> /
+ <filename>bin/yaz_cond&soversion;d.dll</filename></term>
+ <listitem><para>
+ Release/Debug DLL for condition variable utilities (condvar.c).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>lib/yaz_cond&soversion;.lib</filename> /
+ <filename>lib/yaz_cond&soversion;d.lib</filename></term>
+ <listitem><para>
+ Import library for <filename>yaz_cond&soversion;.dll</filename> /
+ <filename>yaz_cond&soversion;d.dll</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz_icu&soversion;.dll</filename> /
+ <filename>bin/yaz_icu&soversion;d.dll</filename></term>
+ <listitem><para>
+ Release/Debug DLL for the ICU wrapper utility.
+ Only build if HAVE_ICU is 1.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>lib/yaz_icu&soversion;.lib</filename> /
+ <filename>lib/yaz_icu&soversion;d.lib</filename></term>
+ <listitem><para>
+ Import library for <filename>yaz_icu&soversion;.dll</filename> /
+ <filename>yaz_icu&soversion;d.dll</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz-ztest.exe</filename></term>
+ <listitem><para>
+ Z39.50 multi-threaded test/example server. It's a WIN32
+ console application.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz-client.exe</filename></term>
+ <listitem><para>
+ &yaz; Z39.50 client application. It's a WIN32 console application.
+ See chapter <link linkend="yaz-client">YAZ client</link> for more
+ information.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/yaz-icu.exe</filename></term>
+ <listitem><para>This program exposes the ICU wrapper library if that
+ is enabled for YAZ. Only if ICU is available this program is
+ build.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/zoomsh.exe</filename></term>
+ <listitem><para>
+ Simple console application implemented on top of the
+ <link linkend="zoom">ZOOM</link> functions.
+ The application is a command line shell that allows you to enter
+ simple commands to perform ZOOM operations.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><filename>bin/zoomtst1.exe</filename>,
+ <filename>bin/zoomtst2.exe</filename>, ..</term>
+ <listitem><para>
+ Several small applications that demonstrates the ZOOM API.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </sect2>
+
+ <sect2 id="installation-linking-yaz-win32">
+ <title>How to make apps using YAZ on WIN32</title>
+ <para>
+ This section will go though the process of linking your WIN32
+ applications with &yaz;.
+ </para>
+ <para>
+ Some people are confused by the fact that we use the nmake
+ tool to build &yaz;. They think they have to do that too - in order
+ to make their WIN32 applications work with &yaz;. The good news is that
+ you don't have to. You can use the integrated environment of
+ Visual Studio if desired for your own application.
+ </para>
+ <para>
+ When setting up a project or Makefile you have to set the following:
+ <variablelist>
+ <varlistentry>
+ <term>include path</term>
+ <listitem><para>
+ Set it to the <filename>include</filename> directory of &yaz;.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>import library <filename>yaz&soversion;.lib</filename></term>
+ <listitem><para>
+ You must link with this library. It's located in the
+ sub directory <filename>lib</filename> of &yaz;.
+ If you want to link with the debug version of &yaz;, you must
+ link against <filename>yaz&soversion;d.lib</filename> instead.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>dynamic link library
+ <filename>yaz&soversion;.dll</filename>
+ </term>
+ <listitem><para>
+ This DLL must be in your execution path when you invoke
+ your application. Specifically, you should distribute this
+ DLL with your application.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </sect2>
+ </sect1>
+ </chapter>
+ <!--
+ ### Still to document:
+ ZOOM_connection_errcode(c)
+ ZOOM_connection_errmsg(c)
+ ZOOM_connection_addinfo(c)
+ ZOOM_connection_addinfo(c)
+ ZOOM_connection_diagset(c);
+ ZOOM_connection_save_apdu_wrbuf
+ ZOOM_diag_str(error)
+ ZOOM_resultset_record_immediate(s, pos)
+ ZOOM_resultset_cache_reset(r)
+ ZOOM_options_set_callback(opt, function, handle)
+ ZOOM_options_create_with_parent2(parent1, parent2)
+ ZOOM_options_getl(opt, name, len)
+ ZOOM_options_setl(opt, name, value, len)
+ ZOOM_options_get_bool(opt, name, defa)
+ ZOOM_options_get_int(opt, name, defa)
+ ZOOM_options_set_int(opt, name, value)
+ -->
+ <chapter id="zoom">
+ <title>ZOOM</title>
+ <para>
+ &zoom; is an acronym for 'Z39.50 Object-Orientation Model' and is
+ an initiative started by Mike Taylor (Mike is from the UK, which
+ explains the peculiar name of the model). The goal of &zoom; is to
+ provide a common Z39.50 client API not bound to a particular
+ programming language or toolkit.
+ </para>
+ <para>
+ From YAZ version 2.1.12, <ulink url="&url.sru;">SRU</ulink> is supported.
+ You can make SRU ZOOM connections by specifying scheme
+ <literal>http://</literal> for the hostname for a connection.
+ The dialect of SRU used is specified by the value of the
+ connection's <literal>sru</literal> option, which may be SRU over
+ HTTP GET (<literal>get</literal>),
+ SRU over HTTP POST (<literal>post</literal>), (SRU over
+ SOAP) (<literal>soap</literal>) or <literal>solr</literal>
+ (<ulink url="&url.solr;">Solr</ulink> Web Service).
+ Using the facility for embedding options in target strings, a
+ connection can be forced to use SRU rather the SRW (the default) by
+ prefixing the target string with <literal>sru=get,</literal>, like this:
+ <literal>sru=get,http://sru.miketaylor.org.uk:80/sru.pl</literal>
+ </para>
+ <para>
+ <ulink url="&url.solr;">Solr</ulink> protocol support was added to
+ YAZ in version 4.1.0, as a dialect of a SRU protocol, since both are
+ HTTP based protocols.
+ </para>
+ <para>
+ The lack of a simple Z39.50 client API for &yaz; has become more
+ and more apparent over time. So when the first &zoom; specification
+ became available,
+ an implementation for &yaz; was quickly developed. For the first time, it is
+ now as easy (or easier!) to develop clients than servers with &yaz;. This
+ chapter describes the &zoom; C binding. Before going further, please
+ reconsider whether C is the right programming language for the job.
+ There are other language bindings available for &yaz;, and still
+ more
+ are in active development. See the
+ <ulink url="&url.zoom;">ZOOM web-site</ulink> for
+ more information.
+ </para>
+ <para>
+ In order to fully understand this chapter you should read and
+ try the example programs <literal>zoomtst1.c</literal>,
+ <literal>zoomtst2.c</literal>, .. in the <literal>zoom</literal>
+ directory.
+ </para>
+ <para>
+ The C language misses features found in object oriented languages
+ such as C++, Java, etc. For example, you'll have to manually,
+ destroy all objects you create, even though you may think of them as
+ temporary. Most objects has a <literal>_create</literal> - and a
+ <literal>_destroy</literal> variant.
+ All objects are in fact pointers to internal stuff, but you don't see
+ that because of typedefs. All destroy methods should gracefully ignore a
+ <literal>NULL</literal> pointer.
+ </para>
+ <para>
+ In each of the sections below you'll find a sub section called
+ protocol behavior, that describes how the API maps to the Z39.50
+ protocol.
+ </para>
+ <sect1 id="zoom-connections">
+ <title>Connections</title>
+ <para>The Connection object is a session with a target.
+ </para>
+ <synopsis>
+ #include <yaz/zoom.h>
+
+ ZOOM_connection ZOOM_connection_new(const char *host, int portnum);
+
+ ZOOM_connection ZOOM_connection_create(ZOOM_options options);
+
+ void ZOOM_connection_connect(ZOOM_connection c, const char *host,
+ int portnum);
+ void ZOOM_connection_destroy(ZOOM_connection c);
+ </synopsis>
+ <para>
+ Connection objects are created with either function
+ <function>ZOOM_connection_new</function> or
+ <function>ZOOM_connection_create</function>.
+ The former creates and automatically attempts to establish a network
+ connection with the target. The latter doesn't establish
+ a connection immediately, thus allowing you to specify options
+ before establishing network connection using the function
+ <function>ZOOM_connection_connect</function>.
+ If the port number, <literal>portnum</literal>, is zero, the
+ <literal>host</literal> is consulted for a port specification.
+ If no port is given, 210 is used. A colon denotes the beginning of
+ a port number in the host string. If the host string includes a
+ slash, the following part specifies a database for the connection.
+ </para>
+ <para>
+ You can prefix the host with a scheme followed by colon. The
+ default scheme is <literal>tcp</literal> (Z39.50 protocol).
+ The scheme <literal>http</literal> selects SRU/get over HTTP by default,
+ but can overridded to use SRU/post, SRW and the Solr protocol.
+ </para>
+ <para>
+ You can prefix the scheme-qualified host-string with one or more
+ comma-separated
+ <literal><parameter>key</parameter>=<parameter>value</parameter></literal>
+ sequences, each of which represents an option to be set into the
+ connection structure <emphasis>before</emphasis> the
+ protocol-level connection is forged and the initialization
+ handshake takes place. This facility can be used to provide
+ authentication credentials, as in host-strings such as:
+ <literal>user=admin,password=halfAm4n,tcp:localhost:8017/db</literal>
+ </para>
+ <para>
+ Connection objects should be destroyed using the function
+ <function>ZOOM_connection_destroy</function>.
+ </para>
+ <synopsis>
+ void ZOOM_connection_option_set(ZOOM_connection c,
+ const char *key, const char *val);
+
+ void ZOOM_connection_option_setl(ZOOM_connection c,
+ const char *key,
+ const char *val, int len);
+
+ const char *ZOOM_connection_option_get(ZOOM_connection c,
+ const char *key);
+ const char *ZOOM_connection_option_getl(ZOOM_connection c,
+ const char *key,
+ int *lenp);
+ </synopsis>
+ <para>
+ The functions <function>ZOOM_connection_option_set</function> and
+ <function>ZOOM_connection_option_setl</function> allows you to
+ set an option given by <parameter>key</parameter> to the value
+ <parameter>value</parameter> for the connection.
+ For <function>ZOOM_connection_option_set</function>, the
+ value is assumed to be a 0-terminated string. Function
+ <function>ZOOM_connection_option_setl</function> specifies a
+ value of a certain size (len).
+ </para>
+ <para>
+ Functions <function>ZOOM_connection_option_get</function> and
+ <function>ZOOM_connection_option_getl</function> returns
+ the value for an option given by <parameter>key</parameter>.
+ </para>
+ <table id="zoom-connection-options" frame="top">
+ <title>ZOOM Connection Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ implementationName</entry><entry>Name of Your client
+ </entry><entry>none</entry></row>
+ <row><entry>
+ user</entry><entry>Authentication user name
+ </entry><entry>none</entry></row>
+ <row><entry>
+ group</entry><entry>Authentication group name
+ </entry><entry>none</entry></row>
+ <row><entry>
+ password</entry><entry>Authentication password.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ authenticationMode</entry><entry>How authentication is encoded.
+ </entry><entry>basic</entry></row>
+ <row><entry>
+ host</entry><entry>Target host. This setting is "read-only".
+ It's automatically set internally when connecting to a target.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ proxy</entry><entry>Proxy host. If set, the logical host
+ is encoded in the otherInfo area of the Z39.50 Init PDU
+ with OID 1.2.840.10003.10.1000.81.1.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ clientIP</entry><entry>Client IP. If set, is
+ encoded in the otherInfo area of a Z39.50 PDU with OID
+ 1.2.840.10003.10.1000.81.3. Holds the original IP addreses
+ of a client. Is used of ZOOM is used in a gateway of some sort.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ async</entry><entry>If true (1) the connection operates in
+ asynchronous operation which means that all calls are non-blocking
+ except
+ <link linkend="zoom.events"><function>ZOOM_event</function></link>.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ maximumRecordSize</entry><entry> Maximum size of single record.
+ </entry><entry>1 MB</entry></row>
+ <row><entry>
+ preferredMessageSize</entry><entry> Maximum size of multiple records.
+ </entry><entry>1 MB</entry></row>
+ <row><entry>
+ lang</entry><entry> Language for negotiation.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ charset</entry><entry> Character set for negotiation.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ serverImplementationId</entry><entry>
+ Implementation ID of server. (The old targetImplementationId
+ option is also supported for the benefit of old applications.)
+ </entry><entry>none</entry></row>
+ <row><entry>
+ targetImplementationName</entry><entry>
+ Implementation Name of server. (The old
+ targetImplementationName option is also supported for the
+ benefit of old applications.)
+ </entry><entry>none</entry></row>
+ <row><entry>
+ serverImplementationVersion</entry><entry>
+ Implementation Version of server. (the old
+ targetImplementationVersion option is also supported for the
+ benefit of old applications.)
+ </entry><entry>none</entry></row>
+ <row><entry>
+ databaseName</entry><entry>One or more database names
+ separated by character plus (<literal>+</literal>), which to
+ be used by subsequent search requests on this Connection.
+ </entry><entry>Default</entry></row>
+ <row><entry>
+ piggyback</entry><entry>True (1) if piggyback should be
+ used in searches; false (0) if not.
+ </entry><entry>1</entry></row>
+ <row><entry>
+ smallSetUpperBound</entry><entry>If hits is less than or equal to this
+ value, then target will return all records using small element set name
+ </entry><entry>0</entry></row>
+ <row><entry>
+ largeSetLowerBound</entry><entry>If hits is greater than this
+ value, the target will return no records.
+ </entry><entry>1</entry></row>
+ <row><entry>
+ mediumSetPresentNumber</entry><entry>This value represents
+ the number of records to be returned as part of a search when when
+ hits is less than or equal to large set lower bound and if hits
+ is greater than small set upper bound.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ smallSetElementSetName</entry><entry>
+ The element set name to be used for small result sets.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ mediumSetElementSetName</entry><entry>
+ The element set name to be for medium-sized result sets.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ init_opt_search, init_opt_present, init_opt_delSet, etc.</entry><entry>
+ After a successful Init, these options may be interrogated to
+ discover whether the server claims to support the specified
+ operations.
+ </entry><entry>none</entry></row>
+ <row>
+ <entry>sru</entry><entry>
+ SRU/Solr transport type. Must be either <literal>soap</literal>,
+ <literal>get</literal>, <literal>post</literal>, or
+ <literal>solr</literal>.
+ </entry><entry>soap</entry></row>
+ <row><entry>
+ sru_version</entry><entry>
+ SRU/SRW version. Should be <literal>1.1</literal>, or
+ <literal>1.2</literal>. This is , prior to connect, the version
+ to offer (highest version). And following connect (in fact
+ first operation), holds the negotiated version with the server
+ (same or lower version).
+ </entry><entry>1.2</entry></row>
+ <row id="zoom.facets.option"><entry>
+ facets</entry><entry>
+ Requested or recommend facets may be given before a search is sent.
+ The value of this setting is described in <xref linkend="facets"/>
+ For inspection of the facets returned, refer to the functions
+ described in <xref linkend="zoom.facets"/>.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ apdulog</entry><entry>
+ If set to a true value such as "1", a log of low-level
+ protocol packets is emitted on standard error stream. This
+ can be very useful for debugging.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ saveAPDU</entry><entry>
+ If set to a true value such as "1", a log of low-level
+ protocol packets is saved. The log can be retrieved by reading
+ option APDU. Setting saveAPDU always has the side effect of
+ resetting the currently saved log. This setting is
+ <emphasis>write-only</emphasis>. If read, NULL will be returned.
+ It is only recognized in
+ <function>ZOOM_connection_option_set</function>.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ APDU</entry><entry>
+ Returns the log of protocol packets. Will be empty if logging
+ is not enabled (see saveAPDU above). This setting is
+ <emphasis>read-only</emphasis>. It is only recognized if used
+ in call to <function>ZOOM_connection_option_get</function> or
+ <function>ZOOM_connection_option_getl</function>.
+ </entry><entry></entry></row>
+ <row><entry>
+ memcached</entry><entry>
+ If given and non-empty,
+ <ulink url="&url.libmemcached;">libMemcached</ulink>
+ will be configured for the connection.
+ This option is inspected by ZOOM when a connection is established.
+ If the <literal>memcached</literal> option is given
+ and YAZ is compiled without libMemcached support, an internal
+ diagnostic (10018) will be thrown.
+ libMemcached support is available for YAZ 5.0.13 or later. If this
+ option is supplied for an earlier version of YAZ, it is
+ <emphasis>ignored</emphasis>.
+ The value of this option is a string passed verbatim to
+ the <function>memcached</function> function part of libMemcached.
+ Refer to man page
+ <ulink url="http://manned.org/memcached.3">memcached(3)</ulink>.
+ Earlier versions of libMemcached
+ do not offer this function. In this case only the option
+ <literal>--server=</literal><replaceable>host</replaceable> may
+ be given (YAZ emulates that part of libMemcached).
+ </entry><entry>none</entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ If either option <literal>lang</literal> or <literal>charset</literal>
+ is set, then
+ <ulink url="&url.z39.50.charneg;">
+ Character Set and Language Negotiation</ulink> is in effect.
+ </para>
+ <synopsis>
+ int ZOOM_connection_error(ZOOM_connection c, const char **cp,
+ const char **addinfo);
+ int ZOOM_connection_error_x(ZOOM_connection c, const char **cp,
+ const char **addinfo, const char **dset);
+ </synopsis>
+ <para>
+ Function <function>ZOOM_connection_error</function> checks for
+ errors for the last operation(s) performed. The function returns
+ zero if no errors occurred; non-zero otherwise indicating the error.
+ Pointers <parameter>cp</parameter> and <parameter>addinfo</parameter>
+ holds messages for the error and additional-info if passed as
+ non-<literal>NULL</literal>. Function
+ <function>ZOOM_connection_error_x</function> is an extended version
+ of <function>ZOOM_connection_error</function> that is capable of
+ returning name of diagnostic set in <parameter>dset</parameter>.
+ </para>
+ <sect2 id="zoom-connection-z39.50">
+ <title>Z39.50 Protocol behavior</title>
+ <para>
+ The calls <function>ZOOM_connection_new</function> and
+ <function>ZOOM_connection_connect</function> establishes a TCP/IP
+ connection and sends an Initialize Request to the target if
+ possible. In addition, the calls waits for an Initialize Response
+ from the target and the result is inspected (OK or rejected).
+ </para>
+ <para>
+ If <literal>proxy</literal> is set then the client will establish
+ a TCP/IP connection with the peer as specified by the
+ <literal>proxy</literal> host and the hostname as part of the
+ connect calls will be set as part of the Initialize Request.
+ The proxy server will then "forward" the PDU's transparently
+ to the target behind the proxy.
+ </para>
+ <para>
+ For the authentication parameters, if option <literal>user</literal>
+ is set and both options <literal>group</literal> and
+ <literal>pass</literal> are unset, then Open style
+ authentication is used (Version 2/3) in which case the username
+ is usually followed by a slash, then by a password.
+ If either <literal>group</literal>
+ or <literal>pass</literal> is set then idPass authentication
+ (Version 3 only) is used. If none of the options are set, no
+ authentication parameters are set as part of the Initialize Request
+ (obviously).
+ </para>
+ <para>
+ When option <literal>async</literal> is 1, it really means that
+ all network operations are postponed (and queued) until the
+ function <literal>ZOOM_event</literal> is invoked. When doing so
+ it doesn't make sense to check for errors after
+ <literal>ZOOM_connection_new</literal> is called since that
+ operation "connecting - and init" is still incomplete and the
+ API cannot tell the outcome (yet).
+ </para>
+ </sect2>
+ <sect2 id="zoom.sru.init.behavior">
+ <title>SRU/Solr Protocol behavior</title>
+ <para>
+ The HTTP based protocols (SRU, SRW, Solr) doesn't feature an
+ Inititialize Request, so the connection phase merely establishes a
+ TCP/IP connection with the HTTP server.
+ </para>
+ <para>Most of the ZOOM connection options do not
+ affect SRU/Solr and they are ignored. However, future versions
+ of &yaz; might honor <literal>implementationName</literal> and
+ put that as part of User-Agent header for HTTP requests.
+ </para>
+ <para>
+ The <literal>charset</literal> is used in the Content-Type header
+ of HTTP requests.
+ </para>
+ <para>
+ Setting <literal>authentcationMode</literal> specifies how
+ authentication parameters are encoded for HTTP. The default is
+ "<literal>basic</literal>" where <literal>user</literal> and
+ <literal>password</literal> are encoded by using HTTP basic
+ authentication.
+ </para>
+ <para>
+ If <literal>authentcationMode</literal> is "<literal>url</literal>", then
+ user and password are encoded in the URL by parameters
+ <literal>x-username</literal> and <literal>x-password</literal> as
+ given by the SRU standard.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="zoom.query">
+ <title>Queries</title>
+ <para>
+ Query objects represents queries.
+ </para>
+ <synopsis>
+ ZOOM_query ZOOM_query_create(void);
+
+ void ZOOM_query_destroy(ZOOM_query q);
+
+ int ZOOM_query_prefix(ZOOM_query q, const char *str);
+
+ int ZOOM_query_cql(ZOOM_query s, const char *str);
+
+ int ZOOM_query_sortby(ZOOM_query q, const char *criteria);
+
+ int ZOOM_query_sortby2(ZOOM_query q, const char *strategy,
+ const char *criteria);
+ </synopsis>
+ <para>
+ Create query objects using <function>ZOOM_query_create</function>
+ and destroy them by calling <function>ZOOM_query_destroy</function>.
+ RPN-queries can be specified in <link linkend="PQF">PQF</link>
+ notation by using the
+ function <function>ZOOM_query_prefix</function>.
+ The <function>ZOOM_query_cql</function> specifies a CQL
+ query to be sent to the server/target.
+ More query types will be added in future versions of &yaz;, such as
+ <link linkend="CCL">CCL</link> to RPN-mapping, native CCL query,
+ etc. In addition to a search, a sort criteria may be set. Function
+ <function>ZOOM_query_sortby</function> enables Z39.50 sorting and
+ it takes sort criteria using the same string notation as
+ yaz-client's <link linkend="sortspec">sort command</link>.
+ </para>
+ <para id="zoom.query.sortby2">
+ <function>ZOOM_query_sortby2</function> is similar to
+ <function>ZOOM_query_sortby</function> but allows a strategy for
+ sorting. The reason for the strategy parameter is that some
+ protocols offers multiple ways of performing sorting.
+ For example, Z39.50 has the standard sort, which is performed after
+ search on an existing result set.
+ It's also possible to use CQL in Z39.50 as the query type and use
+ CQL's SORTBY keyword. Finally, Index Data's
+ Zebra server also allows sorting to be specified as part of RPN (Type 7).
+ </para>
+ <table id="zoom-sort-strategy" frame="top">
+ <title>ZOOM sort strategy</title>
+ <tgroup cols="2">
+ <colspec colwidth="2*" colname="name"/>
+ <colspec colwidth="5*" colname="description"/>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>z39.50</entry><entry>Z39.50 resultset sort</entry>
+ </row>
+ <row>
+ <entry>type7</entry><entry>Sorting embedded in RPN(Type-7)</entry>
+ </row>
+ <row>
+ <entry>cql</entry><entry>CQL SORTBY</entry>
+ </row>
+ <row>
+ <entry>sru11</entry><entry>SRU sortKeys parameter</entry>
+ </row>
+ <row>
+ <entry>solr</entry><entry>Solr sort</entry>
+ </row>
+ <row>
+ <entry>embed</entry><entry>type7 for Z39.50, cql for SRU,
+ solr for Solr protocol</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ <sect1 id="zoom.resultsets"><title>Result sets</title>
+ <para>
+ The result set object is a container for records returned from
+ a target.
+ </para>
+ <synopsis>
+ ZOOM_resultset ZOOM_connection_search(ZOOM_connection, ZOOM_query q);
+
+ ZOOM_resultset ZOOM_connection_search_pqf(ZOOM_connection c,
+ const char *q);
+ void ZOOM_resultset_destroy(ZOOM_resultset r);
+ </synopsis>
+ <para>
+ Function <function>ZOOM_connection_search</function> creates
+ a result set given a connection and query.
+ Destroy a result set by calling
+ <function>ZOOM_resultset_destroy</function>.
+ Simple clients may using PQF only may use function
+ <function>ZOOM_connection_search_pqf</function> in which case
+ creating query objects is not necessary.
+ </para>
+ <synopsis>
+ void ZOOM_resultset_option_set(ZOOM_resultset r,
+ const char *key, const char *val);
+
+ const char *ZOOM_resultset_option_get(ZOOM_resultset r, const char *key);
+
+ size_t ZOOM_resultset_size(ZOOM_resultset r);
+ </synopsis>
+ <para>
+ Functions <function>ZOOM_resultset_options_set</function> and
+ <function>ZOOM_resultset_get</function> sets and gets an option
+ for a result set similar to <function>ZOOM_connection_option_get</function>
+ and <function>ZOOM_connection_option_set</function>.
+ </para>
+ <para>
+ The number of hits also called result-count is returned by
+ function <function>ZOOM_resultset_size</function>.
+ </para>
+ <table id="zoom.resultset.options"
+ frame="top"><title>ZOOM Result set Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="2*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ start</entry><entry>Offset of first record to be
+ retrieved from target. First record has offset 0 unlike the
+ protocol specifications where first record has position 1.
+ This option affects ZOOM_resultset_search and
+ ZOOM_resultset_search_pqf and must be set before any of
+ these functions are invoked. If a range of
+ records must be fetched manually after search,
+ function ZOOM_resultset_records should be used.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ count</entry><entry>Number of records to be retrieved.
+ This option affects ZOOM_resultset_search and
+ ZOOM_resultset_search_pqf and must be set before any of
+ these functions are invoked.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ presentChunk</entry><entry>The number of records to be
+ requested from the server in each chunk (present request). The
+ value 0 means to request all the records in a single chunk.
+ (The old <literal>step</literal>
+ option is also supported for the benefit of old applications.)
+ </entry><entry>0</entry></row>
+ <row><entry>
+ elementSetName</entry><entry>Element-Set name of records.
+ Most targets should honor element set name <literal>B</literal>
+ and <literal>F</literal> for brief and full respectively.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ preferredRecordSyntax</entry><entry>Preferred Syntax, such as
+ <literal>USMARC</literal>, <literal>SUTRS</literal>, etc.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ schema</entry><entry>Schema for retrieval, such as
+ <literal>Gils-schema</literal>, <literal>Geo-schema</literal>, etc.
+ </entry><entry>none</entry></row>
+ <row><entry>
+ setname</entry><entry>Name of Result Set (Result Set ID).
+ If this option isn't set, the ZOOM module will automatically
+ allocate a result set name.
+ </entry><entry>default</entry></row>
+ <row><entry>
+ rpnCharset</entry><entry>Character set for RPN terms.
+ If this is set, ZOOM C will assume that the ZOOM application is
+ running UTF-8. Terms in RPN queries are then converted to the
+ rpnCharset. If this is unset, ZOOM C will not assume any encoding
+ of RPN terms and no conversion is performed.
+ </entry><entry>none</entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ For servers that support Search Info report, the following
+ options may be read using <function>ZOOM_resultset_get</function>.
+ This detailed information is read after a successful search has
+ completed.
+ </para>
+ <para>
+ This information is a list of of items, where each item is
+ information about a term or subquery. All items in the list
+ are prefixed by
+ <literal>SearchResult.</literal><replaceable>no</replaceable>
+ where no presents the item number (0=first, 1=second).
+ Read <literal>searchresult.size</literal> to determine the
+ number of items.
+ </para>
+ <table id="zoom.search.info.report.options"
+ frame="top"><title>Search Info Report Options</title>
+ <tgroup cols="2">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>searchresult.size</entry>
+ <entry>
+ number of search result entries. This option is-nonexistant
+ if no entries are returned by the server.
+ </entry>
+ </row>
+ <row>
+ <entry>searchresult.<replaceable>no</replaceable>.id</entry>
+ <entry>sub query ID</entry>
+ </row>
+ <row>
+ <entry>searchresult.<replaceable>no</replaceable>.count</entry>
+ <entry>result count for item (number of hits)</entry>
+ </row>
+ <row>
+ <entry>searchresult.<replaceable>no</replaceable>.subquery.term</entry>
+ <entry>subquery term</entry>
+ </row>
+ <row>
+ <entry>
+ searchresult.<replaceable>no</replaceable>.interpretation.term
+ </entry>
+ <entry>interpretation term</entry>
+ </row>
+ <row>
+ <entry>
+ searchresult.<replaceable>no</replaceable>.recommendation.term
+ </entry>
+ <entry>recommendation term</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <sect2 id="zoom.z3950.resultset.sort">
+ <title>Z39.50 Result-set Sort</title>
+ <synopsis>
+ void ZOOM_resultset_sort(ZOOM_resultset r,
+ const char *sort_type, const char *sort_spec);
+
+ int ZOOM_resultset_sort1(ZOOM_resultset r,
+ const char *sort_type, const char *sort_spec);
+ </synopsis>
+ <para>
+ <function>ZOOM_resultset_sort</function> and
+ <function>ZOOM_resultset_sort1</function> both sort an existing
+ result-set. The sort_type parameter is not use. Set it to "yaz".
+ The sort_spec is same notation as ZOOM_query_sortby and identical
+ to that offered by yaz-client's
+ <link linkend="sortspec">sort command</link>.
+ </para>
+ <para>
+ These functions only work for Z39.50. Use the more generic utility
+ <link linkend="zoom.query.sortby2">
+ <function>ZOOM_query_sortby2</function></link>
+ for other protocols (and even Z39.50).
+ </para>
+ </sect2>
+ <sect2 id="zoom.z3950.resultset.behavior">
+ <title>Z39.50 Protocol behavior</title>
+ <para>
+ The creation of a result set involves at least a SearchRequest
+ - SearchResponse protocol handshake. Following that, if a sort
+ criteria was specified as part of the query, a SortRequest -
+ SortResponse handshake takes place. Note that it is necessary to
+ perform sorting before any retrieval takes place, so no records will
+ be returned from the target as part of the SearchResponse because these
+ would be unsorted. Hence, piggyback is disabled when sort criteria
+ are set. Following Search - and a possible sort - Retrieval takes
+ place - as one or more Present Requests/Response pairs being
+ transferred.
+ </para>
+ <para>
+ The API allows for two different modes for retrieval. A high level
+ mode which is somewhat more powerful and a low level one.
+ The low level is enabled when searching on a Connection object
+ for which the settings
+ <literal>smallSetUpperBound</literal>,
+ <literal>mediumSetPresentNumber</literal> and
+ <literal>largeSetLowerBound</literal> are set. The low level mode
+ thus allows you to precisely set how records are returned as part
+ of a search response as offered by the Z39.50 protocol.
+ Since the client may be retrieving records as part of the
+ search response, this mode doesn't work well if sorting is used.
+ </para>
+ <para>
+ The high-level mode allows you to fetch a range of records from
+ the result set with a given start offset. When you use this mode
+ the client will automatically use piggyback if that is possible
+ with the target and perform one or more present requests as needed.
+ Even if the target returns fewer records as part of a present response
+ because of a record size limit, etc. the client will repeat sending
+ present requests. As an example, if option <literal>start</literal>
+ is 0 (default) and <literal>count</literal> is 4, and
+ <literal>piggyback</literal> is 1 (default) and no sorting criteria
+ is specified, then the client will attempt to retrieve the 4
+ records as part the search response (using piggyback). On the other
+ hand, if either <literal>start</literal> is positive or if
+ a sorting criteria is set, or if <literal>piggyback</literal>
+ is 0, then the client will not perform piggyback but send Present
+ Requests instead.
+ </para>
+ <para>
+ If either of the options <literal>mediumSetElementSetName</literal> and
+ <literal>smallSetElementSetName</literal> are unset, the value
+ of option <literal>elementSetName</literal> is used for piggyback
+ searches. This means that for the high-level mode you only have
+ to specify one elementSetName option rather than three.
+ </para>
+ </sect2>
+ <sect2 id="zoom.sru.resultset.behavior">
+ <title>SRU Protocol behavior</title>
+ <para>
+ Current version of &yaz; does not take advantage of a result set id
+ returned by the SRU server. Future versions might do, however.
+ Since, the ZOOM driver does not save result set IDs any
+ present (retrieval) is transformed to a SRU SearchRetrieveRequest
+ with same query but, possibly, different offsets.
+ </para>
+ <para>
+ Option <literal>schema</literal> specifies SRU schema
+ for retrieval. However, options <literal>elementSetName</literal> and
+ <literal>preferredRecordSyntax</literal> are ignored.
+ </para>
+ <para>
+ Options <literal>start</literal> and <literal>count</literal>
+ are supported by SRU.
+ The remaining options
+ <literal>piggyback</literal>,
+ <literal>smallSetUpperBound</literal>,
+ <literal>largeSetLowerBound</literal>,
+ <literal>mediumSetPresentNumber</literal>,
+ <literal>mediumSetElementSetName</literal>,
+ <literal>smallSetElementSetName</literal> are
+ unsupported.
+ </para>
+ <para>
+ SRU supports CQL queries, <emphasis>not</emphasis> PQF.
+ If PQF is used, however, the PQF query is transferred anyway
+ using non-standard element <literal>pQuery</literal> in
+ SRU SearchRetrieveRequest.
+ </para>
+ <para>
+ Solr queries has to be done in Solr query format.
+ </para>
+ <para>
+ Unfortunately, SRU or Solr does not define a database setting. Hence,
+ <literal>databaseName</literal> is unsupported and ignored.
+ However, the path part in host parameter for functions
+ <function>ZOOM_connecton_new</function> and
+ <function>ZOOM_connection_connect</function> acts as a
+ database (at least for the &yaz; SRU server).
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="zoom.records">
+ <title>Records</title>
+ <para>
+ A record object is a retrieval record on the client side -
+ created from result sets.
+ </para>
+ <synopsis>
+ void ZOOM_resultset_records(ZOOM_resultset r,
+ ZOOM_record *recs,
+ size_t start, size_t count);
+ ZOOM_record ZOOM_resultset_record(ZOOM_resultset s, size_t pos);
+
+ const char *ZOOM_record_get(ZOOM_record rec, const char *type,
+ size_t *len);
+
+ int ZOOM_record_error(ZOOM_record rec, const char **msg,
+ const char **addinfo, const char **diagset);
+
+ ZOOM_record ZOOM_record_clone(ZOOM_record rec);
+
+ void ZOOM_record_destroy(ZOOM_record rec);
+ </synopsis>
+ <para>
+ References to temporary records are returned by functions
+ <function>ZOOM_resultset_records</function> or
+ <function>ZOOM_resultset_record</function>.
+ </para>
+ <para>
+ If a persistent reference to a record is desired
+ <function>ZOOM_record_clone</function> should be used.
+ It returns a record reference that should be destroyed
+ by a call to <function>ZOOM_record_destroy</function>.
+ </para>
+ <para>
+ A single record is returned by function
+ <function>ZOOM_resultset_record</function> that takes a
+ position as argument. First record has position zero.
+ If no record could be obtained <literal>NULL</literal> is returned.
+ </para>
+ <para>
+ Error information for a record can be checked with
+ <function>ZOOM_record_error</function> which returns non-zero
+ (error code) if record is in error, called <emphasis>Surrogate
+ Diagnostics</emphasis> in Z39.50.
+ </para>
+ <para>
+ Function <function>ZOOM_resultset_records</function> retrieves
+ a number of records from a result set. Parameter <literal>start</literal>
+ and <literal>count</literal> specifies the range of records to
+ be returned. Upon completion array
+ <literal>recs[0], ..recs[count-1]</literal>
+ holds record objects for the records. The array of records
+ <literal>recs</literal> should be allocated prior the call
+ <function>ZOOM_resultset_records</function>. Note that for those
+ records that couldn't be retrieved from the target
+ <literal>recs[ ..]</literal> is set to <literal>NULL</literal>.
+ </para>
+ <para id="zoom.record.get">
+ In order to extract information about a single record,
+ <function>ZOOM_record_get</function> is provided. The
+ function returns a pointer to certain record information. The
+ nature (type) of the pointer depends on the parameter,
+ <parameter>type</parameter>.
+ </para>
+ <para>
+ The <parameter>type</parameter> is a string of the format:
+ </para>
+ <para>
+ <replaceable>format</replaceable>[;charset=<replaceable>from</replaceable>[/<replaceable>opacfrom</replaceable>][,<replaceable>to</replaceable>]][;format=<replaceable>v</replaceable>]
+ </para>
+ <para>
+ where <replaceable>format</replaceable> specifies the format of the
+ returned record, <replaceable>from</replaceable>
+ specifies the character set of the record in its original form
+ (as returned by the server), <replaceable>to</replaceable> specifies
+ the output (returned)
+ character set encoding.
+ If <replaceable>to</replaceable> is omitted UTF-8 is assumed.
+ If charset is not given, then no character set conversion takes place.
+ </para>
+
+ <para>OPAC records may be returned in a different
+ set from the bibliographic MARC record. If this is this the case,
+ <replaceable>opacfrom</replaceable> should be set to the character set
+ of the OPAC record part.
+ </para>
+ <note>
+ <para>
+ Specifying the OPAC record character set requires YAZ 4.1.5 or later.
+ </para>
+ </note>
+ <para>
+ The format argument controls whether record data should be XML
+ pretty-printed (post process operation).
+ It is enabled only if format value <replaceable>v</replaceable> is
+ <literal>1</literal> and the record content is XML well-formed.
+ </para>
+ <para>
+ In addition, for certain types, the length
+ <literal>len</literal> passed will be set to the size in bytes of
+ the returned information.
+ </para>
+ <para>
+ The following are the supported values for <replaceable>form</replaceable>.
+ <variablelist>
+ <varlistentry><term><literal>database</literal></term>
+ <listitem><para>Database of record is returned
+ as a C null-terminated string. Return type
+ <literal>const char *</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>syntax</literal></term>
+ <listitem><para>The transfer syntax of the record is returned
+ as a C null-terminated string containing the symbolic name of
+ the record syntax, e.g. <literal>Usmarc</literal>. Return type
+ is
+ <literal>const char *</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>schema</literal></term>
+ <listitem><para>The schema of the record is returned
+ as a C null-terminated string. Return type is
+ <literal>const char *</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>render</literal></term>
+ <listitem><para>The record is returned in a display friendly
+ format. Upon completion buffer is returned
+ (type <literal>const char *</literal>) and length is stored in
+ <literal>*len</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>raw</literal></term>
+ <listitem><para>The record is returned in the internal
+ YAZ specific format. For GRS-1, Explain, and others, the
+ raw data is returned as type
+ <literal>Z_External *</literal> which is just the type for
+ the member <literal>retrievalRecord</literal> in
+ type <literal>NamePlusRecord</literal>.
+ For SUTRS and octet aligned record (including all MARCs) the
+ octet buffer is returned and the length of the buffer.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>xml</literal></term>
+ <listitem><para>The record is returned in XML if possible.
+ SRU, Solr and Z39.50 records with transfer syntax XML are
+ returned verbatim. MARC records are returned in
+ <ulink url="&url.marcxml;">
+ MARCXML
+ </ulink>
+ (converted from ISO2709 to MARCXML by YAZ).
+ OPAC records are also converted to XML and the
+ bibliographic record is converted to MARCXML (when possible).
+ GRS-1 records are not supported for this form.
+ Upon completion, the XML buffer is returned
+ (type <literal>const char *</literal>) and length is stored in
+ <literal>*len</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>opac</literal></term>
+ <listitem><para>OPAC information for record is returned in XML
+ if an OPAC record is present at the position given. If no
+ OPAC record is present, a NULL pointer is returned.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>txml</literal></term>
+ <listitem><para>The record is returned in TurboMARC if possible.
+ SRU and Z39.50 records with transfer syntax XML are
+ returned verbatim. MARC records are returned in
+ <link linkend="tools.turbomarc">
+ TurboMARC
+ </link>
+ (converted from ISO2709 to TurboMARC by YAZ).
+ Upon completion, the XML buffer is returned
+ (type <literal>const char *</literal>) and length is stored in
+ <literal>*len</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term><literal>json</literal></term>
+ <listitem><para>Like xml, but MARC records are converted to
+ <ulink url="&url.marc_in_json;">MARC-in-JSON</ulink>.
+ </para></listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+ <para>
+ Most
+ <ulink url="&url.marc21;">MARC21</ulink>
+ records uses the
+ <ulink url="&url.marc8;">MARC-8</ulink>
+ character set encoding.
+ An application that wishes to display in Latin-1 would use
+ <screen>
+ render; charset=marc8,iso-8859-1
+ </screen>
+ </para>
+ <sect2 id="zoom.z3950.record.behavior">
+ <title>Z39.50 Protocol behavior</title>
+ <para>
+ The functions <function>ZOOM_resultset_record</function> and
+ <function>ZOOM_resultset_records</function> inspects the client-side
+ record cache. Records not found in cache are fetched using
+ Present.
+ The functions may block (and perform network I/O) - even though option
+ <literal>async</literal> is 1, because they return records objects.
+ (and there's no way to return records objects without retrieving them!).
+ </para>
+ <para>
+ There is a trick, however, in the usage of function
+ <function>ZOOM_resultset_records</function> that allows for
+ delayed retrieval (and makes it non-blocking). By using
+ a null pointer for <parameter>recs</parameter> you're indicating
+ you're not interested in getting records objects
+ <emphasis>now</emphasis>.
+ </para>
+ </sect2>
+ <sect2 id="zoom.sru.record.behavior">
+ <title>SRU/Solr Protocol behavior</title>
+ <para>
+ The ZOOM driver for SRU/Solr treats records returned by a SRU/Solr server
+ as if they where Z39.50 records with transfer syntax XML and
+ no element set name or database name.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="zoom.facets"><title>Facets</title>
+ <para>
+ Facet operations is not part of the official ZOOM specification, but
+ is an Index Data extension for YAZ-based Z39.50 targets,
+ <ulink url="&url.solr;">Solr</ulink> and SRU 2.0 targets.
+
+ Facets may be requestd by the
+ <link linkend="zoom.facets.option">facets</link> option before a
+ search is sent.
+ For inspection of the returned facets, the following functions are
+ available:
+ </para>
+ <synopsis>
+ ZOOM_facet_field *ZOOM_resultset_facets(ZOOM_resultset r);
+
+ ZOOM_facet_field ZOOM_resultset_get_facet_field(ZOOM_resultset r,
+ const char *facet_name);
+
+ ZOOM_facet_field ZOOM_resultset_get_facet_field_by_index(ZOOM_resultset r,
+ int pos);
+
+ size_t ZOOM_resultset_facets_size(ZOOM_resultset r);
+
+ const char *ZOOM_facet_field_name(ZOOM_facet_field facet_field);
+
+ size_t ZOOM_facet_field_term_count(ZOOM_facet_field facet_field);
+
+ const char *ZOOM_facet_field_get_term(ZOOM_facet_field facet_field,
+ size_t idx, int *freq);
+ </synopsis>
+ <para>
+ References to temporary structures are returned by all functions.
+ They are only valid as long the Result set is valid.
+ <function>ZOOM_resultset_get_facet_field</function> or
+ <function>ZOOM_resultset_get_facet_field_by_index</function>.
+ <function>ZOOM_resultset_facets</function>.
+ <function>ZOOM_facet_field_name</function>.
+ <function>ZOOM_facet_field_get_term</function>.
+ </para>
+ <para id="zoom.resultset.get_facet_field">
+ A single Facet field is returned by function
+ <function>ZOOM_resultset_get_facet_field</function> or
+ <function>ZOOM_resultset_get_facet_field_by_index</function> that takes
+ a result set and facet name or positive index respectively. First
+ facet has position zero. If no facet could be obtained (invalid name
+ or index out of bounds) <literal>NULL</literal> is returned.
+ </para>
+ <para id="zoom.resultset.facets">
+ An array of facets field can be returned by
+ <function>ZOOM_resultset_facets</function>. The length of the array is
+ given by <function>ZOOM_resultset_facets_size</function>. The array is
+ zero-based and last entry will be at
+ <function>ZOOM_resultset_facets_size(result_set)</function>-1.
+ </para>
+ <para id="zoom.resultset.facets_names">
+ It is possible to interate over facets by name, by calling
+ <function>ZOOM_resultset_facets_names</function>.
+ This will return an const array of char * where each string can be used
+ as parameter for <function>ZOOM_resultset_get_facet_field</function>.
+ </para>
+ <para>
+ Function <function>ZOOM_facet_field_name</function> gets the request
+ facet name from a returned facet field.
+ </para>
+ <para>
+ Function <function>ZOOM_facet_field_get_term</function> returns the
+ idx'th term and term count for a facet field.
+ Idx must between 0 and
+ <function>ZOOM_facet_field_term_count</function>-1, otherwise the
+ returned reference will be <literal>NULL</literal>. On a valid idx, the
+ value of the freq reference will be the term count.
+ The <literal>freq</literal> parameter must be valid pointer to integer.
+ </para>
+ </sect1>
+ <sect1 id="zoom.scan"><title>Scan</title>
+ <para>
+ This section describes an interface for Scan. Scan is not an
+ official part of the ZOOM model yet. The result of a scan operation
+ is the <literal>ZOOM_scanset</literal> which is a set of terms
+ returned by a target.
+ </para>
+
+ <para>
+ The Scan interface is supported for both Z39.50, SRU and Solr.
+ </para>
+
+ <synopsis>
+ ZOOM_scanset ZOOM_connection_scan(ZOOM_connection c,
+ const char *startpqf);
+
+ ZOOM_scanset ZOOM_connection_scan1(ZOOM_connection c,
+ ZOOM_query q);
+
+ size_t ZOOM_scanset_size(ZOOM_scanset scan);
+
+ const char *ZOOM_scanset_term(ZOOM_scanset scan, size_t pos,
+ size_t *occ, size_t *len);
+
+ const char *ZOOM_scanset_display_term(ZOOM_scanset scan, size_t pos,
+ size_t *occ, size_t *len);
+
+ void ZOOM_scanset_destroy(ZOOM_scanset scan);
+
+ const char *ZOOM_scanset_option_get(ZOOM_scanset scan,
+ const char *key);
+
+ void ZOOM_scanset_option_set(ZOOM_scanset scan, const char *key,
+ const char *val);
+ </synopsis>
+ <para>
+ The scan set is created by function
+ <function>ZOOM_connection_scan</function> which performs a scan
+ operation on the connection using the specified
+ <parameter>startpqf</parameter>.
+ If the operation was successful, the size of the scan set can be
+ retrieved by a call to <function>ZOOM_scanset_size</function>.
+ Like result sets, the items are numbered 0,..size-1.
+ To obtain information about a particular scan term, call function
+ <function>ZOOM_scanset_term</function>. This function takes
+ a scan set offset <literal>pos</literal> and returns a pointer
+ to a <emphasis>raw term</emphasis> or <literal>NULL</literal> if
+ non-present.
+ If present, the <literal>occ</literal> and <literal>len</literal>
+ are set to the number of occurrences and the length
+ of the actual term respectively.
+ <function>ZOOM_scanset_display_term</function> is similar to
+ <function>ZOOM_scanset_term</function> except that it returns
+ the <emphasis>display term</emphasis> rather than the raw term.
+ In a few cases, the term is different from display term. Always
+ use the display term for display and the raw term for subsequent
+ scan operations (to get more terms, next scan result, etc).
+ </para>
+ <para>
+ A scan set may be freed by a call to function
+ <function>ZOOM_scanset_destroy</function>.
+ Functions <function>ZOOM_scanset_option_get</function> and
+ <function>ZOOM_scanset_option_set</function> retrieves and sets
+ an option respectively.
+ </para>
+ <para>
+ The <parameter>startpqf</parameter> is a subset of PQF, namely
+ the Attributes+Term part. Multiple <literal>@attr</literal> can
+ be used. For example to scan in title (complete) phrases:
+ <literallayout>
+ @attr 1=4 @attr 6=2 "science o"
+ </literallayout>
+ </para>
+ <para>
+ The <function>ZOOM_connecton_scan1</function> is a newer and
+ more generic alternative to <function>ZOOM_connection_scan</function>
+ which allows to use both CQL and PQF for Scan.
+ </para>
+ <table frame="top" id="zoom.scanset.options">
+ <title>ZOOM Scan Set Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="2*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ number</entry><entry>Number of Scan Terms requested in next scan.
+ After scan it holds the actual number of terms returned.
+ </entry><entry>20</entry></row>
+ <row><entry>
+ position</entry><entry>Preferred Position of term in response
+ in next scan; actual position after completion of scan.
+ </entry><entry>1</entry></row>
+ <row><entry>
+ stepSize</entry><entry>Step Size
+ </entry><entry>0</entry></row>
+ <row><entry>
+ scanStatus</entry><entry>An integer indicating the Scan Status
+ of last scan.
+ </entry><entry>0</entry></row>
+ <row><entry>
+ rpnCharset</entry><entry>Character set for RPN terms.
+ If this is set, ZOOM C will assume that the ZOOM application is
+ running UTF-8. Terms in RPN queries are then converted to the
+ rpnCharset. If this is unset, ZOOM C will not assume any encoding
+ of RPN terms and no conversion is performed.
+ </entry><entry>none</entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ <sect1 id="zoom.extendedservices">
+ <title>Extended Services</title>
+ <para>
+ ZOOM offers an interface to a subset of the Z39.50 extended services
+ as well as a few privately defined ones:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Z39.50 Item Order (ILL).
+ See <xref linkend="zoom.item.order"/>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Record Update. This allows a client to insert, modify or delete
+ records.
+ See <xref linkend="zoom.record.update"/>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Database Create. This a non-standard feature. Allows a client
+ to create a database.
+ See <xref linkend="zoom.database.create"/>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Database Drop. This a non-standard feature. Allows a client
+ to delete/drop a database.
+ See <xref linkend="zoom.database.drop"/>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Commit operation. This a non-standard feature. Allows a client
+ to commit operations.
+ See <xref linkend="zoom.commit"/>.
+ </para>
+ </listitem>
+ <!-- all the ILL PDU options should go here too -->
+ </itemizedlist>
+ <para>
+ To create an extended service operation a <literal>ZOOM_package</literal>
+ must be created. The operation is a five step operation. The
+ package is created, package is configured by means of options,
+ the package is send, result is inspected (by means of options),
+ the package is destroyed.
+ </para>
+ <synopsis>
+ ZOOM_package ZOOM_connection_package(ZOOM_connection c,
+ ZOOM_options options);
+
+ const char *ZOOM_package_option_get(ZOOM_package p,
+ const char *key);
+ void ZOOM_package_option_set(ZOOM_package p, const char *key,
+ const char *val);
+ void ZOOM_package_send(ZOOM_package p, const char *type);
+
+ void ZOOM_package_destroy(ZOOM_package p);
+ </synopsis>
+ <para>
+ The <function>ZOOM_connection_package</function> creates a
+ package for the connection given using the options specified.
+ </para>
+ <para>
+ Functions <function>ZOOM_package_option_get</function> and
+ <function>ZOOM_package_option_set</function> gets and sets
+ options.
+ </para>
+ <para>
+ <function>ZOOM_package_send</function> sends
+ the package the via connection specified in
+ <function>ZOOM_connection_package</function>.
+ The <parameter>type</parameter> specifies the actual extended service
+ package type to be sent.
+ </para>
+ <table frame="top" id="zoom.extendedservices.options">
+ <title>Extended Service Common Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>package-name</entry>
+ <entry>Extended Service Request package name. Must be specified
+ as part of a request</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>user-id</entry>
+ <entry>User ID of Extended Service Package. Is a request option</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>function</entry>
+ <entry>
+ Function of package - one of <literal>create</literal>,
+ <literal>delete</literal>, <literal>modify</literal>. Is
+ a request option.
+ </entry>
+ <entry><literal>create</literal></entry>
+ </row>
+ <row>
+ <entry>waitAction</entry>
+ <entry>
+ Wait action for package. Possible values:
+ <literal>wait</literal>, <literal>waitIfPossible</literal>,
+ <literal>dontWait</literal> or <literal>dontReturnPackage</literal>.
+ </entry>
+ <entry><literal>waitIfPossible</literal></entry>
+ </row>
+ <row>
+ <entry>targetReference</entry>
+ <entry>
+ Target Reference. This is part of the response as returned
+ by the server. Read it after a successful operation.
+ </entry>
+ <entry><literal>none</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <sect2 id="zoom.item.order">
+ <title>Item Order</title>
+ <para>
+ For Item Order, type must be set to <literal>itemorder</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+
+ <table frame="top" id="zoom.item.order.options">
+ <title>Item Order Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>contact-name</entry>
+ <entry>ILL contact name</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>contact-phone</entry>
+ <entry>ILL contact phone</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>contact-email</entry>
+ <entry>ILL contact email</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>itemorder-item</entry>
+ <entry>Position for item (record) requested. An integer</entry>
+ <entry>1</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+ <sect2 id="zoom.record.update">
+ <title>Record Update</title>
+ <para>
+ For Record Update, type must be set to <literal>update</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+ <table frame="top" id="zoom.record.update.options">
+ <title>Record Update Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>action</entry>
+ <entry>
+ The update action. One of
+ <literal>specialUpdate</literal>,
+ <literal>recordInsert</literal>,
+ <literal>recordReplace</literal>,
+ <literal>recordDelete</literal>,
+ <literal>elementUpdate</literal>.
+ </entry>
+ <entry><literal>specialUpdate (recordInsert for updateVersion=1 which does not support specialUpdate)</literal></entry>
+ </row>
+ <row>
+ <entry>recordIdOpaque</entry>
+ <entry>Opaque Record ID</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>recordIdNumber</entry>
+ <entry>Record ID number</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>record</entry>
+ <entry>The record itself</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>recordOpaque</entry>
+ <entry>Specifies an opaque record which is
+ encoded as an ASN.1 ANY type with the OID as tiven by option
+ <literal>syntax</literal> (see below).
+ Option <literal>recordOpaque</literal> is an alternative
+ to record - and <literal>record</literal> option (above) is
+ ignored if recordOpaque is set. This option is only available in
+ YAZ 3.0.35 and later and is meant to facilitate Updates with
+ servers from OCLC.
+ </entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>syntax</entry>
+ <entry>The record syntax (transfer syntax). Is a string that
+ is a known record syntax.
+ </entry>
+ <entry>no syntax</entry>
+ </row>
+ <row>
+ <entry>databaseName</entry>
+ <entry>Database from connection object</entry>
+ <entry>Default</entry>
+ </row>
+ <row>
+ <entry>correlationInfo.note</entry>
+ <entry>Correlation Info Note (string)</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>correlationInfo.id</entry>
+ <entry>Correlation Info ID (integer)</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>elementSetName</entry>
+ <entry>Element Set for Record</entry>
+ <entry>none</entry>
+ </row>
+ <row>
+ <entry>updateVersion</entry>
+ <entry>Record Update version which holds one of the values
+ 1, 2 or 3. Each version has a distinct OID:
+ 1.2.840.10003.9.5
+ (<ulink url="&url.z39.50.extupdate1;">first version</ulink>) ,
+ 1.2.840.10003.9.5.1
+ (second version) and
+ 1.2.840.10003.9.5.1.1
+ (<ulink url="&url.z39.50.extupdate3;">third and
+ newest version</ulink>).
+ </entry>
+ <entry>3</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+
+ </sect2>
+
+ <sect2 id="zoom.database.create"><title>Database Create</title>
+ <para>
+ For Database Create, type must be set to <literal>create</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+
+ <table frame="top" id="zoom.database.create.options">
+ <title>Database Create Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>databaseName</entry>
+ <entry>Database from connection object</entry>
+ <entry>Default</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+ <sect2 id="zoom.database.drop">
+ <title>Database Drop</title>
+ <para>
+ For Database Drop, type must be set to <literal>drop</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+ <table frame="top" id="zoom.database.drop.options">
+ <title>Database Drop Options</title>
+ <tgroup cols="3">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <colspec colwidth="3*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Option</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>databaseName</entry>
+ <entry>Database from connection object</entry>
+ <entry>Default</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+ <sect2 id="zoom.commit">
+ <title>Commit Operation</title>
+ <para>
+ For Commit, type must be set to <literal>commit</literal> in
+ <function>ZOOM_package_send</function>.
+ </para>
+ </sect2>
+ <sect2 id="zoom.extended.services.behavior">
+ <title>Protocol behavior</title>
+ <para>
+ All the extended services are Z39.50-only.
+ </para>
+ <note>
+ <para>
+ The database create, drop and commit services are privately defined
+ operations.
+ Refer to <filename>esadmin.asn</filename> in YAZ for the ASN.1
+ definitions.
+ </para>
+ </note>
+ </sect2>
+ </sect1>
+ <sect1 id="zoom.options">
+ <title>Options</title>
+ <para>
+ Most &zoom; objects provide a way to specify options to change behavior.
+ From an implementation point of view a set of options is just like
+ an associative array / hash.
+ </para>
+ <synopsis>
+ ZOOM_options ZOOM_options_create(void);
+
+ ZOOM_options ZOOM_options_create_with_parent(ZOOM_options parent);
+
+ void ZOOM_options_destroy(ZOOM_options opt);
+ </synopsis>
+ <synopsis>
+ const char *ZOOM_options_get(ZOOM_options opt, const char *name);
+
+ void ZOOM_options_set(ZOOM_options opt, const char *name,
+ const char *v);
+ </synopsis>
+ <synopsis>
+ typedef const char *(*ZOOM_options_callback)
+ (void *handle, const char *name);
+
+ ZOOM_options_callback
+ ZOOM_options_set_callback(ZOOM_options opt,
+ ZOOM_options_callback c,
+ void *handle);
+ </synopsis>
+ </sect1>
+ <sect1 id="zoom.queryconversions">
+ <title>Query conversions</title>
+ <synopsis>
+ int ZOOM_query_cql2rpn(ZOOM_query s, const char *cql_str,
+ ZOOM_connection conn);
+
+ int ZOOM_query_ccl2rpn(ZOOM_query s, const char *ccl_str,
+ const char *config,
+ int *ccl_error, const char **error_string,
+ int *error_pos);
+ </synopsis>
+ <para>
+ <function>ZOOM_query_cql2rpn</function> translates the CQL string,
+ client-side, into RPN which may be passed to the server.
+ This is useful for server's that don't themselves
+ support CQL, for which <function>ZOOM_query_cql</function> is useless.
+ `conn' is used only as a place to stash diagnostics if compilation
+ fails; if this information is not needed, a null pointer may be used.
+ The CQL conversion is driven by option <literal>cqlfile</literal> from
+ connection conn. This specifies a conversion file (eg pqf.properties)
+ which <emphasis>must</emphasis> be present.
+ </para>
+ <para>
+ <function>ZOOM_query_ccl2rpn</function> translates the CCL string,
+ client-side, into RPN which may be passed to the server.
+ The conversion is driven by the specification given by
+ <literal>config</literal>. Upon completion 0 is returned on success; -1
+ is returned on on failure. Om failure <literal>error_string</literal> and
+ <literal>error_pos</literal> holds error message and position of
+ first error in original CCL string.
+ </para>
+ </sect1>
+ <sect1 id="zoom.events"><title>Events</title>
+ <para>
+ If you're developing non-blocking applications, you have to deal
+ with events.
+ </para>
+ <synopsis>
+ int ZOOM_event(int no, ZOOM_connection *cs);
+ </synopsis>
+ <para>
+ The <function>ZOOM_event</function> executes pending events for
+ a number of connections. Supply the number of connections in
+ <literal>no</literal> and an array of connections in
+ <literal>cs</literal> (<literal>cs[0] ... cs[no-1]</literal>).
+ A pending event could be a sending a search, receiving a response,
+ etc.
+ When an event has occurred for one of the connections, this function
+ returns a positive integer <literal>n</literal> denoting that an event
+ occurred for connection <literal>cs[n-1]</literal>.
+ When no events are pending for the connections, a value of zero is
+ returned.
+ To ensure that all outstanding requests are performed call this function
+ repeatedly until zero is returned.
+ </para>
+ <para>
+ If <function>ZOOM_event</function> returns and returns non-zero, the
+ last event that occurred can be expected.
+ </para>
+ <synopsis>
+ int ZOOM_connection_last_event(ZOOM_connection cs);
+ </synopsis>
+ <para>
+ <function>ZOOM_connection_last_event</function> returns an event type
+ (integer) for the last event.
+ </para>
+
+ <table frame="top" id="zoom.event.ids">
+ <title>ZOOM Event IDs</title>
+ <tgroup cols="2">
+ <colspec colwidth="4*" colname="name"></colspec>
+ <colspec colwidth="7*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Event</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>ZOOM_EVENT_NONE</entry>
+ <entry>No event has occurred</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_CONNECT</entry>
+ <entry>TCP/IP connect has initiated</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_SEND_DATA</entry>
+ <entry>Data has been transmitted (sending)</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_RECV_DATA</entry>
+ <entry>Data has been received)</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_TIMEOUT</entry>
+ <entry>Timeout</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_UNKNOWN</entry>
+ <entry>Unknown event</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_SEND_APDU</entry>
+ <entry>An APDU has been transmitted (sending)</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_RECV_APDU</entry>
+ <entry>An APDU has been received</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_RECV_RECORD</entry>
+ <entry>A result-set record has been received</entry>
+ </row>
+ <row>
+ <entry>ZOOM_EVENT_RECV_SEARCH</entry>
+ <entry>A search result been received</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ </chapter>
+ <chapter id="server">
+ <title>Generic server</title>
+ <sect1 id="server.introduction"><title>Introduction</title>
+ <para>
+ If you aren't into documentation, a good way to learn how the
+ back end interface works is to look at the <filename>backend.h</filename>
+ file. Then, look at the small dummy-server in
+ <filename>ztest/ztest.c</filename>. The <filename>backend.h</filename>
+ file also makes a good reference, once you've chewed your way through
+ the prose of this file.
+ </para>
+ <para>
+ If you have a database system that you would like to make available by
+ means of Z39.50 or SRU, &yaz; basically offers your two options. You
+ can use the APIs provided by the &asn;, &odr;, and &comstack;
+ modules to
+ create and decode PDUs, and exchange them with a client.
+ Using this low-level interface gives you access to all fields and
+ options of the protocol, and you can construct your server as close
+ to your existing database as you like.
+ It is also a fairly involved process, requiring
+ you to set up an event-handling mechanism, protocol state machine,
+ etc. To simplify server implementation, we have implemented a compact
+ and simple, but reasonably full-functioned server-frontend that will
+ handle most of the protocol mechanics, while leaving you to
+ concentrate on your database interface.
+ </para>
+ <note>
+ <para>
+ The backend interface was designed in anticipation of a specific
+ integration task, while still attempting to achieve some degree of
+ generality. We realize fully that there are points where the
+ interface can be improved significantly. If you have specific
+ functions or parameters that you think could be useful, send us a
+ mail (or better, sign on to the mailing list referred to in the
+ top-level README file). We will try to fit good suggestions into future
+ releases, to the extent that it can be done without requiring
+ too many structural changes in existing applications.
+ </para>
+ </note>
+ <note>
+ <para>
+ The &yaz; server does not support XCQL.
+ </para>
+ </note>
+ </sect1>
+ <sect1 id="server.frontend">
+ <title>The Database Frontend</title>
+ <para>
+ We refer to this software as a generic database frontend. Your
+ database system is the <emphasis>backend database</emphasis>, and the
+ interface between the two is called the <emphasis>backend API</emphasis>.
+ The backend API consists of a small number of function handlers and
+ structure definitions. You are required to provide the
+ <function>main()</function> routine for the server (which can be
+ quite simple), as well as a set of handlers to match each of the
+ prototypes.
+ The interface functions that you write can use any mechanism you like
+ to communicate with your database system: You might link the whole
+ thing together with your database application and access it by
+ function calls; you might use IPC to talk to a database server
+ somewhere; or you might link with third-party software that handles
+ the communication for you (like a commercial database client library).
+ At any rate, the handlers will perform the tasks of:
+ </para>
+ <itemizedlist>
+ <listitem><para>
+ Initialization.
+ </para></listitem>
+ <listitem><para>
+ Searching.
+ </para></listitem>
+ <listitem><para>
+ Fetching records.
+ </para></listitem>
+ <listitem><para>
+ Scanning the database index (optional - if you wish to implement SCAN).
+ </para></listitem>
+ <listitem><para>
+ Extended Services (optional).
+ </para></listitem>
+ <listitem><para>
+ Result-Set Delete (optional).
+ </para></listitem>
+ <listitem><para>
+ Result-Set Sort (optional).
+ </para></listitem>
+ <listitem><para>
+ Return Explain for SRU (optional).
+ </para></listitem>
+ </itemizedlist>
+ <para>
+ (more functions will be added in time to support as much of
+ Z39.50-1995 as possible).
+ </para>
+ </sect1>
+ <sect1 id="server.backend">
+ <title>The Backend API</title>
+ <para>
+ The header file that you need to use the interface are in the
+ <filename>include/yaz</filename> directory. It's called
+ <filename>backend.h</filename>. It will include other files from
+ the <filename>include/yaz</filename> directory, so you'll
+ probably want to use the -I option of your compiler to tell it
+ where to find the files. When you run
+ <literal>make</literal> in the top-level &yaz; directory,
+ everything you need to create your server is to link with the
+ <filename>lib/libyaz.la</filename> library.
+ </para>
+ </sect1>
+ <sect1 id="server.main">
+ <title>Your main() Routine</title>
+ <para>
+ As mentioned, your <function>main()</function> routine can be quite brief.
+ If you want to initialize global parameters, or read global configuration
+ tables, this is the place to do it. At the end of the routine, you should
+ call the function
+ </para>
+ <synopsis>
+int statserv_main(int argc, char **argv,
+ bend_initresult *(*bend_init)(bend_initrequest *r),
+ void (*bend_close)(void *handle));
+ </synopsis>
+ <para>
+ The third and fourth arguments are pointers to handlers. Handler
+ <function>bend_init</function> is called whenever the server receives
+ an Initialize Request, so it serves as a Z39.50 session initializer. The
+ <function>bend_close</function> handler is called when the session is
+ closed.
+ </para>
+ <para>
+ <function>statserv_main</function> will establish listening sockets
+ according to the parameters given. When connection requests are received,
+ the event handler will typically <function>fork()</function> and
+ create a sub-process to handle a new connection.
+ Alternatively the server may be setup to create threads for each
+ connection.
+ If you do use global variables and forking, you should be aware, then,
+ that these cannot be shared between associations, unless you explicitly
+ disable forking by command line parameters.
+ </para>
+ <para>
+ The server provides a mechanism for controlling some of its behavior
+ without using command-line options. The function
+ </para>
+ <synopsis>
+ statserv_options_block *statserv_getcontrol(void);
+ </synopsis>
+ <para>
+ will return a pointer to a <literal>struct statserv_options_block</literal>
+ describing the current default settings of the server. The structure
+ contains these elements:
+ <variablelist>
+ <varlistentry>
+ <term><literal>int dynamic</literal></term>
+ <listitem><para>
+ A boolean value, which determines whether the server
+ will fork on each incoming request (TRUE), or not (FALSE). Default is
+ TRUE. This flag is only read by UNIX-based servers (WIN32 based servers
+ doesn't fork).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>int threads</literal></term>
+ <listitem><para>
+ A boolean value, which determines whether the server
+ will create a thread on each incoming request (TRUE), or not (FALSE).
+ Default is FALSE. This flag is only read by UNIX-based servers
+ that offer POSIX Threads support.
+ WIN32-based servers always operate in threaded mode.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>int inetd</literal></term>
+ <listitem><para>
+ A boolean value, which determines whether the server
+ will operates under a UNIX INET daemon (inetd). Default is FALSE.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char logfile[ODR_MAXNAME+1]</literal></term>
+ <listitem><para>File for diagnostic output ("": stderr).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char apdufile[ODR_MAXNAME+1]</literal></term>
+ <listitem><para>
+ Name of file for logging incoming and outgoing APDUs
+ ("": don't log APDUs, "-":
+ <literal>stderr</literal>).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char default_listen[1024]</literal></term>
+ <listitem><para>Same form as the command-line specification of
+ listener address. "": no default listener address.
+ Default is to listen at "tcp:@:9999". You can only
+ specify one default listener address in this fashion.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>enum oid_proto default_proto;</literal></term>
+ <listitem><para>Either <literal>PROTO_Z3950</literal> or
+ <literal>PROTO_SR</literal>.
+ Default is <literal>PROTO_Z39_50</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>int idle_timeout;</literal></term>
+ <listitem><para>Maximum session idle-time, in minutes. Zero indicates
+ no (infinite) timeout. Default is 15 minutes.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>int maxrecordsize;</literal></term>
+ <listitem><para>Maximum permissible record (message) size. Default
+ is 64 MB. This amount of memory will only be allocated if a
+ client requests a very large amount of records in one operation
+ (or a big record).
+ Set it to a lower number if you are worried about resource
+ consumption on your host system.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char configname[ODR_MAXNAME+1]</literal></term>
+ <listitem><para>Passed to the backend when a new connection is received.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>char setuid[ODR_MAXNAME+1]</literal></term>
+ <listitem><para>Set user id to the user specified, after binding
+ the listener addresses.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>void (*bend_start)(struct statserv_options_block *p)</literal>
+ </term>
+ <listitem><para>Pointer to function which is called after the
+ command line options have been parsed - but before the server
+ starts listening.
+ For forked UNIX servers this handler is called in the mother
+ process; for threaded servers this handler is called in the
+ main thread.
+ The default value of this pointer is NULL in which case it
+ isn't invoked by the frontend server.
+ When the server operates as an NT service this handler is called
+ whenever the service is started.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>void (*bend_stop)(struct statserv_options_block *p)</literal>
+ </term>
+ <listitem><para>Pointer to function which is called whenever the server
+ has stopped listening for incoming connections. This function pointer
+ has a default value of NULL in which case it isn't called.
+ When the server operates as an NT service this handler is called
+ whenever the service is stopped.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>void *handle</literal></term>
+ <listitem><para>User defined pointer (default value NULL).
+ This is a per-server handle that can be used to specify "user-data".
+ Do not confuse this with the session-handle as returned by bend_init.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ The pointer returned by <literal>statserv_getcontrol</literal> points to
+ a static area. You are allowed to change the contents of the structure,
+ but the changes will not take effect before you call
+ </para>
+ <synopsis>
+void statserv_setcontrol(statserv_options_block *block);
+ </synopsis>
+ <note>
+ <para>
+ that you should generally update this structure before calling
+ <function>statserv_main()</function>.
+ </para>
+ </note>
+ </sect1>
+ <sect1 id="server.backendfunctions">
+ <title>The Backend Functions</title>
+ <para>
+ For each service of the protocol, the backend interface declares one or
+ two functions. You are required to provide implementations of the
+ functions representing the services that you wish to implement.
+ </para>
+ <sect2 id="server.init">
+ <title>Init</title>
+ <synopsis>
+bend_initresult (*bend_init)(bend_initrequest *r);
+ </synopsis>
+ <para>
+ This handler is called once for each new connection request, after
+ a new process/thread has been created, and an Initialize Request has
+ been received from the client. The pointer to the
+ <function>bend_init</function> handler is passed in the call to
+ <function>statserv_start</function>.
+ </para>
+ <para>
+ This handler is also called when operating in SRU mode - when
+ a connection has been made (even though SRU does not offer
+ this service).
+ </para>
+ <para>
+ Unlike previous versions of YAZ, the <function>bend_init</function> also
+ serves as a handler that defines the Z39.50 services that the backend
+ wish to support. Pointers to <emphasis>all</emphasis> service handlers,
+ including search - and fetch must be specified here in this handler.
+ </para>
+ <para>
+ The request - and result structures are defined as
+ </para>
+ <synopsis>
+typedef struct bend_initrequest
+{
+ /** \brief user/name/password to be read */
+ Z_IdAuthentication *auth;
+ /** \brief encoding stream (for results) */
+ ODR stream;
+ /** \brief printing stream */
+ ODR print;
+ /** \brief decoding stream (use stream for results) */
+ ODR decode;
+ /** \brief reference ID */
+ Z_ReferenceId *referenceId;
+ /** \brief peer address of client */
+ char *peer_name;
+
+ /** \brief character set and language negotiation
+
+ see include/yaz/z-charneg.h
+ */
+ Z_CharSetandLanguageNegotiation *charneg_request;
+
+ /** \brief character negotiation response */
+ Z_External *charneg_response;
+
+ /** \brief character set (encoding) for query terms
+
+ This is NULL by default. It should be set to the native character
+ set that the backend assumes for query terms */
+ char *query_charset;
+
+ /** \brief whehter query_charset also applies to recors
+
+ Is 0 (No) by default. Set to 1 (yes) if records is in the same
+ character set as queries. If in doubt, use 0 (No).
+ */
+ int records_in_same_charset;
+
+ char *implementation_id;
+ char *implementation_name;
+ char *implementation_version;
+
+ /** \brief Z39.50 sort handler */
+ int (*bend_sort)(void *handle, bend_sort_rr *rr);
+ /** \brief SRU/Z39.50 search handler */
+ int (*bend_search)(void *handle, bend_search_rr *rr);
+ /** \brief SRU/Z39.50 fetch handler */
+ int (*bend_fetch)(void *handle, bend_fetch_rr *rr);
+ /** \brief SRU/Z39.50 present handler */
+ int (*bend_present)(void *handle, bend_present_rr *rr);
+ /** \brief Z39.50 extended services handler */
+ int (*bend_esrequest) (void *handle, bend_esrequest_rr *rr);
+ /** \brief Z39.50 delete result set handler */
+ int (*bend_delete)(void *handle, bend_delete_rr *rr);
+ /** \brief Z39.50 scan handler */
+ int (*bend_scan)(void *handle, bend_scan_rr *rr);
+ /** \brief Z39.50 segment facility handler */
+ int (*bend_segment)(void *handle, bend_segment_rr *rr);
+ /** \brief SRU explain handler */
+ int (*bend_explain)(void *handle, bend_explain_rr *rr);
+ /** \brief SRU scan handler */
+ int (*bend_srw_scan)(void *handle, bend_scan_rr *rr);
+ /** \brief SRU record update handler */
+ int (*bend_srw_update)(void *handle, bend_update_rr *rr);
+
+ /** \brief whether named result sets are supported (0=disable, 1=enable) */
+ int named_result_sets;
+} bend_initrequest;
+
+typedef struct bend_initresult
+{
+ int errcode; /* 0==OK */
+ char *errstring; /* system error string or NULL */
+ void *handle; /* private handle to the backend module */
+} bend_initresult;
+ </synopsis>
+ <para>
+ In general, the server frontend expects that the
+ <literal>bend_*result</literal> pointer that you return is valid at
+ least until the next call to a <literal>bend_* function</literal>.
+ This applies to all of the functions described herein. The parameter
+ structure passed to you in the call belongs to the server frontend, and
+ you should not make assumptions about its contents after the current
+ function call has completed. In other words, if you want to retain any
+ of the contents of a request structure, you should copy them.
+ </para>
+ <para>
+ The <literal>errcode</literal> should be zero if the initialization of
+ the backend went well. Any other value will be interpreted as an error.
+ The <literal>errstring</literal> isn't used in the current version, but
+ one option would be to stick it in the initResponse as a VisibleString.
+ The <literal>handle</literal> is the most important parameter. It should
+ be set to some value that uniquely identifies the current session to
+ the backend implementation. It is used by the frontend server in any
+ future calls to a backend function.
+ The typical use is to set it to point to a dynamically allocated state
+ structure that is private to your backend module.
+ </para>
+ <para>
+ The <literal>auth</literal> member holds the authentication information
+ part of the Z39.50 Initialize Request. Interpret this if your serves
+ requires authentication.
+ </para>
+ <para>
+ The members <literal>peer_name</literal>,
+ <literal>implementation_id</literal>,
+ <literal>implementation_name</literal> and
+ <literal>implementation_version</literal> holds
+ DNS of client, ID of implementor, name
+ of client (Z39.50) implementation - and version.
+ </para>
+ <para>
+ The <literal>bend_</literal> - members are set to NULL when
+ <function>bend_init</function> is called. Modify the pointers by
+ setting them to point to backend functions.
+ </para>
+ </sect2>
+ <sect2 id="server.search.retrieve">
+ <title>Search and Retrieve</title>
+ <para>
+ We now describe the handlers that are required to support search -
+ and retrieve. You must support two functions - one for search - and one
+ for fetch (retrieval of one record). If desirable you can provide a
+ third handler which is called when a present request is received which
+ allows you to optimize retrieval of multiple-records.
+ </para>
+ <synopsis>
+int (*bend_search) (void *handle, bend_search_rr *rr);
+
+typedef struct {
+ char *setname; /* name to give to this set */
+ int replace_set; /* replace set, if it already exists */
+ int num_bases; /* number of databases in list */
+ char **basenames; /* databases to search */
+ Z_ReferenceId *referenceId;/* reference ID */
+ Z_Query *query; /* query structure */
+ ODR stream; /* encode stream */
+ ODR decode; /* decode stream */
+ ODR print; /* print stream */
+
+ bend_request request;
+ bend_association association;
+ int *fd;
+ int hits; /* number of hits */
+ int errcode; /* 0==OK */
+ char *errstring; /* system error string or NULL */
+ Z_OtherInformation *search_info; /* additional search info */
+ char *srw_sortKeys; /* holds SRU/SRW sortKeys info */
+ char *srw_setname; /* holds SRU/SRW generated resultsetID */
+ int *srw_setnameIdleTime; /* holds SRU/SRW life-time */
+ int estimated_hit_count; /* if hit count is estimated */
+ int partial_resultset; /* if result set is partial */
+} bend_search_rr;
+ </synopsis>
+ <para>
+ The <function>bend_search</function> handler is a fairly close
+ approximation of a protocol Z39.50 Search Request - and Response PDUs
+ The <literal>setname</literal> is the resultSetName from the protocol.
+ You are required to establish a mapping between the set name and whatever
+ your backend database likes to use.
+ Similarly, the <literal>replace_set</literal> is a boolean value
+ corresponding to the resultSetIndicator field in the protocol.
+ <literal>num_bases/basenames</literal> is a length of/array of character
+ pointers to the database names provided by the client.
+ The <literal>query</literal> is the full query structure as defined in
+ the protocol ASN.1 specification.
+ It can be either of the possible query types, and it's up to you to
+ determine if you can handle the provided query type.
+ Rather than reproduce the C interface here, we'll refer you to the
+ structure definitions in the file
+ <filename>include/yaz/z-core.h</filename>. If you want to look at the
+ attributeSetId OID of the RPN query, you can either match it against
+ your own internal tables, or you can use the <link linkend="tools.oid">
+ OID tools</link>.
+ </para>
+ <para>
+ The structure contains a number of hits, and an
+ <literal>errcode/errstring</literal> pair. If an error occurs
+ during the search, or if you're unhappy with the request, you should
+ set the errcode to a value from the BIB-1 diagnostic set. The value
+ will then be returned to the user in a nonsurrogate diagnostic record
+ in the response. The <literal>errstring</literal>, if provided, will
+ go in the addinfo field. Look at the protocol definition for the
+ defined error codes, and the suggested uses of the addinfo field.
+ </para>
+ <para>
+ The <function>bend_search</function> handler is also called when
+ the frontend server receives a SRU SearchRetrieveRequest.
+ For SRU, a CQL query is usually provided by the client.
+ The CQL query is available as part of <literal>Z_Query</literal>
+ structure (note that CQL is now part of Z39.50 via an external).
+ To support CQL in existing implementations that only do Type-1,
+ we refer to the CQL-to-PQF tool described
+ <link linkend="cql.to.pqf">here</link>.
+ </para>
+ <para>
+ To maintain backwards compatibility, the frontend server
+ of yaz always assume that error codes are BIB-1 diagnostics.
+ For SRU operation, a Bib-1 diagnostic code is mapped to
+ SRU diagnostic.
+ </para>
+ <synopsis>
+int (*bend_fetch) (void *handle, bend_fetch_rr *rr);
+
+typedef struct bend_fetch_rr {
+ char *setname; /* set name */
+ int number; /* record number */
+ Z_ReferenceId *referenceId;/* reference ID */
+ Odr_oid *request_format; /* format, transfer syntax (OID) */
+ Z_RecordComposition *comp; /* Formatting instructions */
+ ODR stream; /* encoding stream - memory source if req */
+ ODR print; /* printing stream */
+
+ char *basename; /* name of database that provided record */
+ int len; /* length of record or -1 if structured */
+ char *record; /* record */
+ int last_in_set; /* is it? */
+ Odr_oid *output_format; /* response format/syntax (OID) */
+ int errcode; /* 0==success */
+ char *errstring; /* system error string or NULL */
+ int surrogate_flag; /* surrogate diagnostic */
+ char *schema; /* string record schema input/output */
+} bend_fetch_rr;
+ </synopsis>
+ <para>
+ The frontend server calls the <function>bend_fetch</function> handler
+ when it needs database records to fulfill a Z39.50 Search Request, a
+ Z39.50 Present Request or a SRU SearchRetrieveRequest.
+ The <literal>setname</literal> is simply the name of the result set
+ that holds the reference to the desired record.
+ The <literal>number</literal> is the offset into the set (with 1
+ being the first record in the set). The <literal>format</literal> field
+ is the record format requested by the client (See
+ <xref linkend="tools.oid"/>).
+ A value of NULL for <literal>format</literal> indicates that the
+ client did not request a specific format.
+ The <literal>stream</literal> argument is an &odr; stream which
+ should be used for allocating space for structured data records.
+ The stream will be reset when all records have been assembled, and
+ the response package has been transmitted.
+ For unstructured data, the backend is responsible for maintaining a
+ static or dynamic buffer for the record between calls.
+ </para>
+ <para>
+ If a SRU SearchRetrieveRequest is received by the frontend server,
+ the <literal>referenceId</literal> is NULL and the
+ <literal>format</literal> (transfer syntax) is the OID for XML.
+ The schema for SRU is stored in both the
+ <literal>Z_RecordComposition</literal>
+ structure and <literal>schema</literal> (simple string).
+ </para>
+ <para>
+ In the structure, the <literal>basename</literal> is the name of the
+ database that holds the
+ record. <literal>len</literal> is the length of the record returned, in
+ bytes, and <literal>record</literal> is a pointer to the record.
+ <literal>last_in_set</literal> should be nonzero only if the record
+ returned is the last one in the given result set.
+ <literal>errcode</literal> and <literal>errstring</literal>, if
+ given, will be interpreted as a global error pertaining to the
+ set, and will be returned in a non-surrogate-diagnostic.
+ If you wish to return the error as a surrogate-diagnostic
+ (local error) you can do this by setting
+ <literal>surrogate_flag</literal> to 1 also.
+ </para>
+ <para>
+ If the <literal>len</literal> field has the value -1, then
+ <literal>record</literal> is assumed to point to a constructed data
+ type. The <literal>format</literal> field will be used to determine
+ which encoder should be used to serialize the data.
+ </para>
+ <note>
+ <para>
+ If your backend generates structured records, it should use
+ <function>odr_malloc()</function> on the provided stream for allocating
+ data: This allows the frontend server to keep track of the record sizes.
+ </para>
+ </note>
+ <para>
+ The <literal>format</literal> field is mapped to an object identifier
+ in the direct reference of the resulting EXTERNAL representation
+ of the record.
+ </para>
+ <note>
+ <para>
+ The current version of &yaz; only supports the direct reference mode.
+ </para>
+ </note>
+ <synopsis>
+int (*bend_present) (void *handle, bend_present_rr *rr);
+
+typedef struct {
+ char *setname; /* set name */
+ int start;
+ int number; /* record number */
+ Odr_oid *format; /* format, transfer syntax (OID) */
+ Z_ReferenceId *referenceId;/* reference ID */
+ Z_RecordComposition *comp; /* Formatting instructions */
+ ODR stream; /* encoding stream - memory source if required */
+ ODR print; /* printing stream */
+ bend_request request;
+ bend_association association;
+
+ int hits; /* number of hits */
+ int errcode; /* 0==OK */
+ char *errstring; /* system error string or NULL */
+} bend_present_rr;
+ </synopsis>
+ <para>
+ The <function>bend_present</function> handler is called when
+ the server receives a Z39.50 Present Request.
+ The <literal>setname</literal>,
+ <literal>start</literal> and <literal>number</literal> is the
+ name of the result set - start position - and number of records to
+ be retrieved respectively. <literal>format</literal> and
+ <literal>comp</literal> is the preferred transfer syntax and element
+ specifications of the present request.
+ </para>
+ <para>
+ Note that this is handler serves as a supplement for
+ <function>bend_fetch</function> and need not to be defined in order to
+ support search - and retrieve.
+ </para>
+ </sect2>
+ <sect2 id="server.delete">
+ <title>Delete</title>
+ <para>
+ For back-ends that supports delete of a result set only one handler
+ must be defined.
+ </para>
+ <synopsis>
+int (*bend_delete)(void *handle, bend_delete_rr *rr);
+
+typedef struct bend_delete_rr {
+ int function;
+ int num_setnames;
+ char **setnames;
+ Z_ReferenceId *referenceId;
+ int delete_status; /* status for the whole operation */
+ int *statuses; /* status each set - indexed as setnames */
+ ODR stream;
+ ODR print;
+} bend_delete_rr;
+ </synopsis>
+ <note>
+ <para>
+ The delete set function definition is rather primitive, mostly because
+ we have had no practical need for it as of yet. If someone wants
+ to provide a full delete service, we'd be happy to add the
+ extra parameters that are required. Are there clients out there
+ that will actually delete sets they no longer need?
+ </para>
+ </note>
+ </sect2>
+ <sect2 id="server.scan">
+ <title>Scan</title>
+ <para>
+ For servers that wish to offer the scan service one handler
+ must be defined.
+ </para>
+ <synopsis>
+int (*bend_scan)(void *handle, bend_scan_rr *rr);
+
+typedef enum {
+ BEND_SCAN_SUCCESS, /* ok */
+ BEND_SCAN_PARTIAL /* not all entries could be found */
+} bend_scan_status;
+
+typedef struct bend_scan_rr {
+ int num_bases; /* number of elements in databaselist */
+ char **basenames; /* databases to search */
+ Odr_oid *attributeset;
+ Z_ReferenceId *referenceId; /* reference ID */
+ Z_AttributesPlusTerm *term;
+ ODR stream; /* encoding stream - memory source if required */
+ ODR print; /* printing stream */
+
+ int *step_size; /* step size */
+ int term_position; /* desired index of term in result list/returned */
+ int num_entries; /* number of entries requested/returned */
+
+ /* scan term entries. The called handler does not have
+ to allocate this. Size of entries is num_entries (see above) */
+ struct scan_entry *entries;
+ bend_scan_status status;
+ int errcode;
+ char *errstring;
+ char *scanClause; /* CQL scan clause */
+ char *setname; /* Scan in result set (NULL if omitted) */
+} bend_scan_rr;
+ </synopsis>
+ <para>
+ This backend server handles both Z39.50 scan
+ and SRU scan. In order for a handler to distinguish between SRU (CQL) scan
+ Z39.50 Scan , it must check for a non-NULL value of
+ <literal>scanClause</literal>.
+ </para>
+ <note>
+ <para>
+ if designed today, it would be a choice using a union or similar,
+ but that would break binary compatibility with existing servers.
+ </para>
+ </note>
+ </sect2>
+ </sect1>
+ <sect1 id="server.invocation">
+ <title>Application Invocation</title>
+ <para>
+ The finished application has the following
+ invocation syntax (by way of <function>statserv_main()</function>):
+ </para>
+ &gfs-synopsis;
+ <para>
+ The options are:
+ &gfs-options;
+ </para>
+ <para>
+ A listener specification consists of a transport mode followed by a
+ colon (:) followed by a listener address. The transport mode is
+ either <literal>tcp</literal>, <literal>unix:</literal> or
+ <literal>ssl</literal>.
+ </para>
+ <para>
+ For TCP and SSL, an address has the form
+ </para>
+ <synopsis>
+ hostname | IP-number [: portnumber]
+ </synopsis>
+ <para>
+ The port number defaults to 210 (standard Z39.50 port).
+ </para>
+ <para>
+ For UNIX, the address is the filename of socket.
+ </para>
+ <para>
+ For TCP/IP and SSL, the special hostnames <literal>@</literal>,
+ maps to <literal>IN6ADDR_ANY_INIT</literal> with
+ IPV4 binding as well (bindv6only=0),
+ The special hostname <literal>@4</literal> binds to
+ <literal>INADDR_ANY</literal> (IPV4 only listener).
+ The special hostname <literal>@6</literal> binds to
+ <literal>IN6ADDR_ANY_INIT</literal> with bindv6only=1 (IPV6 only listener).
+ </para>
+ <example id="server.example.running.unix">
+ <title>Running the GFS on Unix</title>
+ <para>
+ Assuming the server application <replaceable>appname</replaceable> is
+ started as root, the following will make it listen on port 210.
+ The server will change identity to <literal>nobody</literal>
+ and write its log to <filename>/var/log/app.log</filename>.
+ <screen>
+ application -l /var/log/app.log -u nobody tcp:@:210
+ </screen>
+ </para>
+ <para>
+ The server will accept Z39.50 requests and offer SRU service on port 210.
+ </para>
+ </example>
+ <example id="server.example.apache.sru">
+ <title>Setting up Apache as SRU Frontend</title>
+ <para>
+ If you use <ulink url="&url.apache;">Apache</ulink>
+ as your public web server and want to offer HTTP port 80
+ access to the YAZ server on 210, you can use the
+ <ulink url="&url.apache.directive.proxypass;">
+ <literal>ProxyPass</literal></ulink>
+ directive.
+ If you have virtual host
+ <literal>srw.mydomain</literal> you can use the following directives
+ in Apache's httpd.conf:
+ <screen>
+ <VirtualHost *>
+ ErrorLog /home/srw/logs/error_log
+ TransferLog /home/srw/logs/access_log
+ ProxyPass / http://srw.mydomain:210/
+ </VirtualHost>
+ </screen>
+ </para>
+ <para>
+ The above for the Apache 1.3 series.
+ </para>
+ </example>
+ <example id="server.example.local.access">
+ <title>Running a server with local access only</title>
+ <para>
+ Servers that is only being accessed from the local host should listen
+ on UNIX file socket rather than a Internet socket. To listen on
+ <filename>/tmp/mysocket</filename> start the server as follows:
+ <screen>
+ application unix:/tmp/mysocket
+ </screen>
+ </para>
+ </example>
+ </sect1>
+ <sect1 id="server.vhosts">
+ <title>GFS Configuration and Virtual Hosts</title>
+ &gfs-virtual;
+ </sect1>
+ </chapter>
+ <chapter id="asn">
+ <title>The Z39.50 ASN.1 Module</title>
+ <sect1 id="asn.introduction">
+ <title>Introduction</title>
+ <para>
+ The &asn; module provides you with a set of C struct definitions for the
+ various PDUs of the Z39.50 protocol, as well as for the complex types
+ appearing within the PDUs. For the primitive data types, the C
+ representation often takes the form of an ordinary C language type,
+ such as <literal>Odr_int</literal> which is equivalent to an integral
+ C integer. For ASN.1 constructs that have no direct
+ representation in C, such as general octet strings and bit strings,
+ the &odr; module (see section <link linkend="odr">The ODR Module</link>)
+ provides auxiliary definitions.
+ </para>
+ <para>
+ The &asn; module is located in sub directory <filename>z39.50</filename>.
+ There you'll find C files that implements encoders and decoders for the
+ Z39.50 types. You'll also find the protocol definitions:
+ <filename>z3950v3.asn</filename>, <filename>esupdate.asn</filename>,
+ and others.
+ </para>
+ </sect1>
+ <sect1 id="asn.preparing">
+ <title>Preparing PDUs</title>
+ <para>
+ A structure representing a complex ASN.1 type doesn't in itself contain the
+ members of that type. Instead, the structure contains
+ <emphasis>pointers</emphasis> to the members of the type.
+ This is necessary, in part, to allow a mechanism for specifying which
+ of the optional structure (SEQUENCE) members are present, and which
+ are not. It follows that you will need to somehow provide space for
+ the individual members of the structure, and set the pointers to
+ refer to the members.
+ </para>
+ <para>
+ The conversion routines don't care how you allocate and maintain your
+ C structures - they just follow the pointers that you provide.
+ Depending on the complexity of your application, and your personal
+ taste, there are at least three different approaches that you may take
+ when you allocate the structures.
+ </para>
+ <para>
+ You can use static or automatic local variables in the function that
+ prepares the PDU. This is a simple approach, and it provides the most
+ efficient form of memory management. While it works well for flat
+ PDUs like the InitReqest, it will generally not be sufficient for say,
+ the generation of an arbitrarily complex RPN query structure.
+ </para>
+ <para>
+ You can individually create the structure and its members using the
+ <function>malloc(2)</function> function. If you want to ensure that
+ the data is freed when it is no longer needed, you will have to
+ define a function that individually releases each member of a
+ structure before freeing the structure itself.
+ </para>
+ <para>
+ You can use the <function>odr_malloc()</function> function (see
+ <xref linkend="odr.use"/> for details). When you use
+ <function>odr_malloc()</function>, you can release all of the
+ allocated data in a single operation, independent of any pointers and
+ relations between the data. <function>odr_malloc()</function> is based on a
+ "nibble-memory"
+ scheme, in which large portions of memory are allocated, and then
+ gradually handed out with each call to <function>odr_malloc()</function>.
+ The next time you call <function>odr_reset()</function>, all of the
+ memory allocated since the last call is recycled for future use (actually,
+ it is placed on a free-list).
+ </para>
+ <para>
+ You can combine all of the methods described here. This will often be
+ the most practical approach. For instance, you might use
+ <function>odr_malloc()</function> to allocate an entire structure and
+ some of its elements, while you leave other elements pointing to global
+ or per-session default variables.
+ </para>
+ <para>
+ The &asn; module provides an important aid in creating new PDUs. For
+ each of the PDU types (say, <function>Z_InitRequest</function>), a
+ function is provided that allocates and initializes an instance of
+ that PDU type for you. In the case of the InitRequest, the function is
+ simply named <function>zget_InitRequest()</function>, and it sets up
+ reasonable default value for all of the mandatory members. The optional
+ members are generally initialized to null pointers. This last aspect
+ is very important: it ensures that if the PDU definitions are
+ extended after you finish your implementation (to accommodate
+ new versions of the protocol, say), you won't get into trouble with
+ uninitialized pointers in your structures. The functions use
+ <function>odr_malloc()</function> to
+ allocate the PDUs and its members, so you can free everything again with a
+ single call to <function>odr_reset()</function>. We strongly recommend
+ that you use the <literal>zget_*</literal>
+ functions whenever you are preparing a PDU (in a C++ API, the
+ <literal>zget_</literal>
+ functions would probably be promoted to constructors for the
+ individual types).
+ </para>
+ <para>
+ The prototype for the individual PDU types generally look like this:
+ </para>
+ <synopsis>
+ Z_<type> *zget_<type>(ODR o);
+ </synopsis>
+ <para>
+ eg.:
+ </para>
+ <synopsis>
+ Z_InitRequest *zget_InitRequest(ODR o);
+ </synopsis>
+ <para>
+ The &odr; handle should generally be your encoding stream, but it
+ needn't be.
+ </para>
+ <para>
+ As well as the individual PDU functions, a function
+ <function>zget_APDU()</function> is provided, which allocates
+ a top-level Z-APDU of the type requested:
+ </para>
+ <synopsis>
+ Z_APDU *zget_APDU(ODR o, int which);
+ </synopsis>
+ <para>
+ The <varname>which</varname> parameter is (of course) the discriminator
+ belonging to the <varname>Z_APDU</varname> <literal>CHOICE</literal> type.
+ All of the interface described here is provided by the &asn; module, and
+ you access it through the <filename>proto.h</filename> header file.
+ </para>
+ </sect1>
+ <sect1 id="asn.external">
+ <title>EXTERNAL Data</title>
+ <para>
+ In order to achieve extensibility and adaptability to different
+ application domains, the new version of the protocol defines many
+ structures outside of the main ASN.1 specification, referencing them
+ through ASN.1 EXTERNAL constructs. To simplify the construction and
+ access to the externally referenced data, the &asn; module defines a
+ specialized version of the EXTERNAL construct, called
+ <literal>Z_External</literal>.It is defined thus:
+ </para>
+ <screen>
+typedef struct Z_External
+{
+ Odr_oid *direct_reference;
+ int *indirect_reference;
+ char *descriptor;
+ enum
+ {
+ /* Generic types */
+ Z_External_single = 0,
+ Z_External_octet,
+ Z_External_arbitrary,
+
+ /* Specific types */
+ Z_External_SUTRS,
+ Z_External_explainRecord,
+ Z_External_resourceReport1,
+ Z_External_resourceReport2
+
+ ...
+
+ } which;
+ union
+ {
+ /* Generic types */
+ Odr_any *single_ASN1_type;
+ Odr_oct *octet_aligned;
+ Odr_bitmask *arbitrary;
+
+ /* Specific types */
+ Z_SUTRS *sutrs;
+ Z_ExplainRecord *explainRecord;
+ Z_ResourceReport1 *resourceReport1;
+ Z_ResourceReport2 *resourceReport2;
+
+ ...
+
+ } u;
+} Z_External;
+ </screen>
+ <para>
+ When decoding, the &asn; module will attempt to determine which
+ syntax describes the data by looking at the reference fields
+ (currently only the direct-reference). For ASN.1 structured data, you
+ need only consult the <literal>which</literal> field to determine the
+ type of data. You can the access the data directly through the union.
+ When constructing data for encoding, you set the union pointer to point
+ to the data, and set the <literal>which</literal> field accordingly.
+ Remember also to set the direct (or indirect) reference to the correct
+ OID for the data type.
+ For non-ASN.1 data such as MARC records, use the
+ <literal>octet_aligned</literal> arm of the union.
+ </para>
+ <para>
+ Some servers return ASN.1 structured data values (eg. database
+ records) as BER-encoded records placed in the
+ <literal>octet-aligned</literal> branch of the EXTERNAL CHOICE.
+ The ASN-module will <emphasis>not</emphasis> automatically decode
+ these records. To help you decode the records in the application, the
+ function
+ </para>
+ <screen>
+ Z_ext_typeent *z_ext_gettypebyref(const oid *oid);
+ </screen>
+ <para>
+ Can be used to retrieve information about the known, external data
+ types. The function return a pointer to a static area, or NULL, if no
+ match for the given direct reference is found. The
+ <literal>Z_ext_typeent</literal>
+ is defined as:
+ </para>
+ <screen>
+typedef struct Z_ext_typeent
+{
+ int oid[OID_SIZE]; /* the direct-reference OID. */
+ int what; /* discriminator value for the external CHOICE */
+ Odr_fun fun; /* decoder function */
+} Z_ext_typeent;
+ </screen>
+ <para>
+ The <literal>what</literal> member contains the
+ <literal>Z_External</literal> union discriminator value for the
+ given type: For the SUTRS record syntax, the value would be
+ <literal>Z_External_sutrs</literal>.
+ The <literal>fun</literal> member contains a pointer to the
+ function which encodes/decodes the given type. Again, for the SUTRS
+ record syntax, the value of <literal>fun</literal> would be
+ <literal>z_SUTRS</literal> (a function pointer).
+ </para>
+ <para>
+ If you receive an EXTERNAL which contains an octet-string value that
+ you suspect of being an ASN.1-structured data value, you can use
+ <literal>z_ext_gettypebyref</literal> to look for the provided
+ direct-reference.
+ If the return value is different from NULL, you can use the provided
+ function to decode the BER string (see <xref linkend="odr.use"/>
+ ).
+ </para>
+ <para>
+ If you want to <emphasis>send</emphasis> EXTERNALs containing
+ ASN.1-structured values in the occtet-aligned branch of the CHOICE, this
+ is possible too. However, on the encoding phase, it requires a somewhat
+ involved juggling around of the various buffers involved.
+ </para>
+ <para>
+ If you need to add new, externally defined data types, you must update
+ the struct above, in the source file <filename>prt-ext.h</filename>, as
+ well as the encoder/decoder in the file <filename>prt-ext.c</filename>.
+ When changing the latter, remember to update both the
+ <literal>arm</literal> arrary and the list
+ <literal>type_table</literal>, which drives the CHOICE biasing that
+ is necessary to tell the different, structured types apart
+ on decoding.
+ </para>
+ <note>
+ <para>
+ Eventually, the EXTERNAL processing will most likely
+ automatically insert the correct OIDs or indirect-refs. First,
+ however, we need to determine how application-context management
+ (specifically the presentation-context-list) should fit into the
+ various modules.
+ </para>
+ </note>
+ </sect1>
+ <sect1 id="asn.pdu">
+ <title>PDU Contents Table</title>
+ <para>
+ We include, for reference, a listing of the fields of each top-level
+ PDU, as well as their default settings.
+ </para>
+ <table frame="top" id="asn.default.initialize.request">
+ <title>Default settings for PDU Initialize Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
+ </entry></row>
+ <row><entry>
+ options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
+ </entry></row>
+ <row><entry>
+ preferredMessageSize</entry><entry>Odr_int</entry><entry>30*1024
+ </entry></row>
+ <row><entry>
+ maximumRecordSize</entry><entry>Odr_int</entry><entry>30*1024
+ </entry></row>
+ <row><entry>
+ idAuthentication</entry><entry>Z_IdAuthentication</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ implementationId</entry><entry>char*</entry><entry>"81"
+ </entry></row>
+ <row><entry>
+ implementationName</entry><entry>char*</entry><entry>"YAZ"
+ </entry></row>
+ <row><entry>
+ implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
+ </entry></row>
+ <row><entry>
+ userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.initialize.response">
+ <title>Default settings for PDU Initialize Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ protocolVersion</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
+ </entry></row>
+ <row><entry>
+ options</entry><entry>Odr_bitmask</entry><entry>Empty bitmask
+ </entry></row>
+ <row><entry>
+ preferredMessageSize</entry><entry>Odr_int</entry><entry>30*1024
+ </entry></row>
+ <row><entry>
+ maximumRecordSize</entry><entry>Odr_int</entry><entry>30*1024
+ </entry></row>
+ <row><entry>
+ result</entry><entry>Odr_bool</entry><entry>TRUE
+ </entry></row>
+ <row><entry>
+ implementationId</entry><entry>char*</entry><entry>"id)"
+ </entry></row>
+ <row><entry>
+ implementationName</entry><entry>char*</entry><entry>"YAZ"
+ </entry></row>
+ <row><entry>
+ implementationVersion</entry><entry>char*</entry><entry>YAZ_VERSION
+ </entry></row>
+ <row><entry>
+ userInformationField</entry><entry>Z_UserInformation</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.search.request">
+ <title>Default settings for PDU Search Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ smallSetUpperBound</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ largeSetLowerBound</entry><entry>Odr_int</entry><entry>1
+ </entry></row>
+ <row><entry>
+ mediumSetPresentNumber</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ replaceIndicator</entry><entry>Odr_bool</entry><entry>TRUE
+ </entry></row>
+ <row><entry>
+ resultSetName</entry><entry>char *</entry><entry>"default"
+ </entry></row>
+ <row><entry>
+ num_databaseNames</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ databaseNames</entry><entry>char **</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ smallSetElementSetNames</entry><entry>Z_ElementSetNames
+ </entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ mediumSetElementSetNames</entry><entry>Z_ElementSetNames
+ </entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ query</entry><entry>Z_Query</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ additionalSearchInfo</entry><entry>Z_OtherInformation
+ </entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.search.response">
+ <title>Default settings for PDU Search Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resultCount</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ nextResultSetPosition</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ searchStatus</entry><entry>Odr_bool</entry><entry>TRUE
+ </entry></row>
+ <row><entry>
+ resultSetStatus</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ presentStatus</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ records</entry><entry>Z_Records</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ additionalSearchInfo</entry>
+ <entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.present.request">
+ <title>Default settings for PDU Present Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resultSetId</entry><entry>char*</entry><entry>"default"
+ </entry></row>
+ <row><entry>
+ resultSetStartPoint</entry><entry>Odr_int</entry><entry>1
+ </entry></row>
+ <row><entry>
+ numberOfRecordsRequested</entry><entry>Odr_int</entry><entry>10
+ </entry></row>
+ <row><entry>
+ num_ranges</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ additionalRanges</entry><entry>Z_Range</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ recordComposition</entry><entry>Z_RecordComposition</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ preferredRecordSyntax</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ maxSegmentCount</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ maxRecordSize</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ maxSegmentSize</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.present.response">
+ <title>Default settings for PDU Present Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ nextResultSetPosition</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ presentStatus</entry><entry>Odr_int</entry><entry>Z_PresentStatus_success
+ </entry></row>
+ <row><entry>
+ records</entry><entry>Z_Records</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.delete.result.set.request">
+ <title>Default settings for Delete Result Set Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>referenceId
+ </entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ deleteFunction</entry><entry>Odr_int</entry><entry>Z_DeleteResultSetRequest_list
+ </entry></row>
+ <row><entry>
+ num_ids</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ resultSetList</entry><entry>char**</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.delete.result.set.response">
+ <title>Default settings for Delete Result Set Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ deleteOperationStatus</entry><entry>Odr_int</entry>
+ <entry>Z_DeleteStatus_success</entry></row>
+ <row><entry>
+ num_statuses</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ deleteListStatuses</entry><entry>Z_ListStatus**</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ numberNotDeleted</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ num_bulkStatuses</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ bulkStatuses</entry><entry>Z_ListStatus</entry><entry>NUL
+ L</entry></row>
+ <row><entry>
+ deleteMessage</entry><entry>char*</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.scan.request">
+ <title>Default settings for Scan Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ num_databaseNames</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ databaseNames</entry><entry>char**</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ attributeSet</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ termListAndStartPoint</entry><entry>Z_AttributesPlus...
+ </entry><entry>NULL</entry></row>
+ <row><entry>
+ stepSize</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ numberOfTermsRequested</entry><entry>Odr_int</entry><entry>20
+ </entry></row>
+ <row><entry>
+ preferredPositionInResponse</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.scan.response">
+ <title>Default settings for Scan Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ stepSize</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ scanStatus</entry><entry>Odr_int</entry><entry>Z_Scan_success
+ </entry></row>
+ <row><entry>
+ numberOfEntriesReturned</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ positionOfTerm</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ entries</entry><entry>Z_ListEntris</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ attributeSet</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.trigger.resource.control.request">
+ <title>Default settings for Trigger Resource Control Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ requestedAction</entry><entry>Odr_int</entry><entry>
+ Z_TriggerResourceCtrl_resou..
+ </entry></row>
+ <row><entry>
+ prefResourceReportFormat</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resultSetWanted</entry><entry>Odr_bool</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.resource.control.request">
+ <title>Default settings for Resource Control Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ suspendedFlag</entry><entry>Odr_bool</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resourceReport</entry><entry>Z_External</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ partialResultsAvailable</entry><entry>Odr_int</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ responseRequired</entry><entry>Odr_bool</entry><entry>FALSE
+ </entry></row>
+ <row><entry>
+ triggeredRequestFlag</entry><entry>Odr_bool</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.resource.control.response">
+ <title>Default settings for Resource Control Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ continueFlag</entry><entry>bool_t</entry><entry>TRUE
+ </entry></row>
+ <row><entry>
+ resultSetWanted</entry><entry>bool_t</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.access.control.request">
+ <title>Default settings for Access Control Request</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ which</entry><entry>enum</entry><entry>Z_AccessRequest_simpleForm;
+ </entry></row>
+ <row><entry>
+ u</entry><entry>union</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.access.control.response">
+ <title>Default settings for Access Control Response</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ which</entry><entry>enum</entry><entry>Z_AccessResponse_simpleForm
+ </entry></row>
+ <row><entry>
+ u</entry><entry>union</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ diagnostic</entry><entry>Z_DiagRec</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.segment">
+ <title>Default settings for Segment</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ numberOfRecordsReturned</entry><entry>Odr_int</entry><entry>value=0
+ </entry></row>
+ <row><entry>
+ num_segmentRecords</entry><entry>Odr_int</entry><entry>0
+ </entry></row>
+ <row><entry>
+ segmentRecords</entry><entry>Z_NamePlusRecord</entry><entry>NULL
+ </entry></row>
+ <row><entry>otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ <table frame="top" id="asn.default.close">
+ <title>Default settings for Close</title>
+ <tgroup cols="3">
+ <colspec colwidth="7*" colname="field"></colspec>
+ <colspec colwidth="5*" colname="type"></colspec>
+ <colspec colwidth="7*" colname="value"></colspec>
+ <thead>
+ <row>
+ <entry>Field</entry>
+ <entry>Type</entry>
+ <entry>Default Value</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row><entry>
+ referenceId</entry><entry>Z_ReferenceId</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ closeReason</entry><entry>Odr_int</entry><entry>Z_Close_finished
+ </entry></row>
+ <row><entry>
+ diagnosticInformation</entry><entry>char*</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resourceReportFormat</entry><entry>Odr_oid</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ resourceFormat</entry><entry>Z_External</entry><entry>NULL
+ </entry></row>
+ <row><entry>
+ otherInfo</entry><entry>Z_OtherInformation</entry><entry>NULL
+ </entry></row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ </chapter>
+ <chapter id="soap">
+ <title>SOAP and SRU</title>
+ <sect1 id="soap.introduction">
+ <title>Introduction</title>
+ <para>
+ &yaz; uses a very simple implementation of
+ <ulink url="&url.soap;">SOAP</ulink> that only,
+ currenly, supports what is sufficient to offer SRU SOAP functionality.
+ The implementation uses the
+ <ulink url="&url.libxml2.api.tree;">tree API</ulink> of
+ libxml2 to encode and decode SOAP packages.
+ </para>
+ <para>
+ Like the Z39.50 ASN.1 module, the &yaz; SRU implementation uses
+ simple C structs to represent SOAP packages as well as
+ HTTP packages.
+ </para>
+ </sect1>
+ <sect1 id="soap.http">
+ <title>HTTP</title>
+ <para>
+ &yaz; only offers HTTP as transport carrier for SOAP, but it is
+ relatively easy to change that.
+ </para>
+ <para>
+ The following definition of <literal>Z_GDU</literal> (Generic Data
+ Unit) allows for both HTTP and Z39.50 in one packet.
+ </para>
+ <synopsis>
+#include <yaz/zgdu.h>
+
+#define Z_GDU_Z3950 1
+#define Z_GDU_HTTP_Request 2
+#define Z_GDU_HTTP_Response 3
+typedef struct {
+ int which;
+ union {
+ Z_APDU *z3950;
+ Z_HTTP_Request *HTTP_Request;
+ Z_HTTP_Response *HTTP_Response;
+ } u;
+} Z_GDU ;
+ </synopsis>
+ <para>
+ The corresponding Z_GDU encoder/decoder is <function>z_GDU</function>.
+ The <literal>z3950</literal> is any of the known BER encoded Z39.50
+ APDUs.
+ <literal>HTTP_Request</literal> and <literal>HTTP_Response</literal>
+ is the HTTP Request and Response respectively.
+ </para>
+ </sect1>
+ <sect1 id="soap.xml">
+ <title>SOAP Packages</title>
+ <para>
+ Every SOAP package in &yaz; is represented as follows:
+ <synopsis>
+#include <yaz/soap.h>
+
+typedef struct {
+ char *fault_code;
+ char *fault_string;
+ char *details;
+} Z_SOAP_Fault;
+
+typedef struct {
+ int no;
+ char *ns;
+ void *p;
+} Z_SOAP_Generic;
+
+#define Z_SOAP_fault 1
+#define Z_SOAP_generic 2
+#define Z_SOAP_error 3
+typedef struct {
+ int which;
+ union {
+ Z_SOAP_Fault *fault;
+ Z_SOAP_Generic *generic;
+ Z_SOAP_Fault *soap_error;
+ } u;
+ const char *ns;
+} Z_SOAP;
+ </synopsis>
+ </para>
+ <para>
+ The <literal>fault</literal> and <literal>soap_error</literal>
+ arms represent both a SOAP fault - struct
+ <literal>Z_SOAP_Fault</literal>. Any other generic
+ (valid) package is represented by <literal>Z_SOAP_Generic</literal>.
+ </para>
+ <para>
+ The <literal>ns</literal> as part of <literal>Z_SOAP</literal>
+ is the namespace for SOAP itself and reflects the SOAP
+ version. For version 1.1 it is
+ <literal>http://schemas.xmlsoap.org/soap/envelope/</literal>,
+ for version 1.2 it is
+ <literal>http://www.w3.org/2001/06/soap-envelope</literal>.
+ </para>
+ <synopsis>
+int z_soap_codec(ODR o, Z_SOAP **pp,
+ char **content_buf, int *content_len,
+ Z_SOAP_Handler *handlers);
+ </synopsis>
+ <para>
+ The <literal>content_buf</literal> and <literal>content_len</literal>
+ is XML buffer and length of buffer respectively.
+ </para>
+ <para>
+ The <literal>handlers</literal> is a list of SOAP codec
+ handlers - one handler for each service namespace. For SRU SOAP, the
+ namespace would be <literal>http://www.loc.gov/zing/srw/v1.0/</literal>.
+ </para>
+ <para>
+ When decoding, the <function>z_soap_codec</function>
+ inspects the XML content
+ and tries to match one of the services namespaces of the
+ supplied handlers. If there is a match a handler function
+ is invoked which decodes that particular SOAP package.
+ If successful, the returned <literal>Z_SOAP</literal> package will be
+ of type <literal>Z_SOAP_Generic</literal>.
+ Member <literal>no</literal> is
+ set the offset of handler that matched; <literal>ns</literal>
+ is set to namespace of matching handler; the void pointer
+ <literal>p</literal> is set to the C data structure assocatiated
+ with the handler.
+ </para>
+ <para>
+ When a NULL namespace is met (member <literal>ns</literal> bwlow),
+ that specifies end-of-list.
+ </para>
+ <para>
+ Each handler is defined as follows:
+ <synopsis>
+typedef struct {
+ char *ns;
+ void *client_data;
+ Z_SOAP_fun f;
+} Z_SOAP_Handler;
+ </synopsis>
+ The <literal>ns</literal> is namespace of service associated with
+ handler <literal>f</literal>. <literal>client_data</literal>
+ is user-defined data which is passed to handler.
+ </para>
+ <para>
+ The prototype for a SOAP service handler is:
+ <synopsis>
+int handler(ODR o, void * ptr, void **handler_data,
+ void *client_data, const char *ns);
+ </synopsis>
+ The <parameter>o</parameter> specifies the mode (decode/encode)
+ as usual. The second argument, <parameter>ptr</parameter>,
+ is a libxml2 tree node pointer (<literal>xmlNodePtr</literal>)
+ and is a pointer to the <literal>Body</literal> element
+ of the SOAP package. The <parameter>handler_data</parameter>
+ is an opaque pointer to a C definitions associated with the
+ SOAP service. <parameter>client_data</parameter> is the pointer
+ which was set as part of the <literal>Z_SOAP_handler</literal>.
+ Finally, <parameter>ns</parameter> the service namespace.
+ </para>
+ </sect1>
+ <sect1 id="soap.srw">
+ <title>SRU</title>
+ <para>
+ SRU SOAP is just one implementation of a SOAP handler as described
+ in the previous section.
+ The encoder/decoder handler for SRU is defined as
+ follows:
+ <synopsis>
+#include <yaz/srw.h>
+
+int yaz_srw_codec(ODR o, void * pptr,
+ Z_SRW_GDU **handler_data,
+ void *client_data, const char *ns);
+ </synopsis>
+ Here, <literal>Z_SRW_GDU</literal> is either
+ searchRetrieveRequest or a searchRetrieveResponse.
+ </para>
+ <note>
+ <para>
+ The xQuery and xSortKeys are not handled yet by
+ the SRW implementation of &yaz;. Explain is also missing.
+ Future versions of &yaz; will include these features.
+ </para>
+ </note>
+ <para>
+ The definition of searchRetrieveRequest is:
+ <synopsis>
+typedef struct {
+
+#define Z_SRW_query_type_cql 1
+#define Z_SRW_query_type_xcql 2
+#define Z_SRW_query_type_pqf 3
+ int query_type;
+ union {
+ char *cql;
+ char *xcql;
+ char *pqf;
+ } query;
+
+#define Z_SRW_sort_type_none 1
+#define Z_SRW_sort_type_sort 2
+#define Z_SRW_sort_type_xSort 3
+ int sort_type;
+ union {
+ char *none;
+ char *sortKeys;
+ char *xSortKeys;
+ } sort;
+ int *startRecord;
+ int *maximumRecords;
+ char *recordSchema;
+ char *recordPacking;
+ char *database;
+} Z_SRW_searchRetrieveRequest;
+ </synopsis>
+ Please observe that data of type xsd:string is represented
+ as a char pointer (<literal>char *</literal>). A null pointer
+ means that the element is absent.
+ Data of type xsd:integer is representd as a pointer to
+ an int (<literal>int *</literal>). Again, a null pointer
+ us used for absent elements.
+ </para>
+ <para>
+ The SearchRetrieveResponse has the following definition.
+ <synopsis>
+typedef struct {
+ int * numberOfRecords;
+ char * resultSetId;
+ int * resultSetIdleTime;
+
+ Z_SRW_record *records;
+ int num_records;
+
+ Z_SRW_diagnostic *diagnostics;
+ int num_diagnostics;
+ int *nextRecordPosition;
+} Z_SRW_searchRetrieveResponse;
+ </synopsis>
+ The <literal>num_records</literal> and <literal>num_diagnostics</literal>
+ is number of returned records and diagnostics respectively and also
+ correspond to the "size of" arrays <literal>records</literal>
+ and <literal>diagnostics</literal>.
+ </para>
+ <para>
+ A retrieval record is defined as follows:
+ <synopsis>
+typedef struct {
+ char *recordSchema;
+ char *recordData_buf;
+ int recordData_len;
+ int *recordPosition;
+} Z_SRW_record;
+ </synopsis>
+ The record data is defined as a buffer of some length so that
+ data can be of any type. SRW 1.0 currenly doesn't allow for this
+ (only XML), but future versions might do.
+ </para>
+ <para>
+ And, a diagnostic as:
+ <synopsis>
+typedef struct {
+ int *code;
+ char *details;
+} Z_SRW_diagnostic;
+ </synopsis>
+ </para>
+ </sect1>
+ </chapter>
+ <chapter id="tools">
+ <title>Supporting Tools</title>
+ <para>
+ In support of the service API - primarily the ASN module, which
+ provides the pro-grammatic interface to the Z39.50 APDUs, &yaz; contains
+ a collection of tools that support the development of applications.
+ </para>
+ <sect1 id="tools.query">
+ <title>Query Syntax Parsers</title>
+ <para>
+ Since the type-1 (RPN) query structure has no direct, useful string
+ representation, every origin application needs to provide some form of
+ mapping from a local query notation or representation to a
+ <token>Z_RPNQuery</token> structure. Some programmers will prefer to
+ construct the query manually, perhaps using
+ <function>odr_malloc()</function> to simplify memory management.
+ The &yaz; distribution includes three separate, query-generating tools
+ that may be of use to you.
+ </para>
+ <sect2 id="PQF">
+ <title>Prefix Query Format</title>
+ <para>
+ Since RPN or reverse polish notation is really just a fancy way of
+ describing a suffix notation format (operator follows operands), it
+ would seem that the confusion is total when we now introduce a prefix
+ notation for RPN. The reason is one of simple laziness - it's somewhat
+ simpler to interpret a prefix format, and this utility was designed
+ for maximum simplicity, to provide a baseline representation for use
+ in simple test applications and scripting environments (like Tcl). The
+ demonstration client included with YAZ uses the PQF.
+ </para>
+ <note>
+ <para>
+ The PQF have been adopted by other parties developing Z39.50
+ software. It is often referred to as Prefix Query Notation
+ - PQN.
+ </para>
+ </note>
+ <para>
+ The PQF is defined by the pquery module in the YAZ library.
+ There are two sets of function that have similar behavior. First
+ set operates on a PQF parser handle, second set doesn't. First set
+ set of functions are more flexible than the second set. Second set
+ is obsolete and is only provided to ensure backwards compatibility.
+ </para>
+ <para>
+ First set of functions all operate on a PQF parser handle:
+ </para>
+ <synopsis>
+ #include <yaz/pquery.h>
+
+ YAZ_PQF_Parser yaz_pqf_create(void);
+
+ void yaz_pqf_destroy(YAZ_PQF_Parser p);
+
+ Z_RPNQuery *yaz_pqf_parse(YAZ_PQF_Parser p, ODR o, const char *qbuf);
+
+ Z_AttributesPlusTerm *yaz_pqf_scan(YAZ_PQF_Parser p, ODR o,
+ Odr_oid **attributeSetId, const char *qbuf);
+
+ int yaz_pqf_error(YAZ_PQF_Parser p, const char **msg, size_t *off);
+ </synopsis>
+ <para>
+ A PQF parser is created and destructed by functions
+ <function>yaz_pqf_create</function> and
+ <function>yaz_pqf_destroy</function> respectively.
+ Function <function>yaz_pqf_parse</function> parses query given
+ by string <literal>qbuf</literal>. If parsing was successful,
+ a Z39.50 RPN Query is returned which is created using ODR stream
+ <literal>o</literal>. If parsing failed, a NULL pointer is
+ returned.
+ Function <function>yaz_pqf_scan</function> takes a scan query in
+ <literal>qbuf</literal>. If parsing was successful, the function
+ returns attributes plus term pointer and modifies
+ <literal>attributeSetId</literal> to hold attribute set for the
+ scan request - both allocated using ODR stream <literal>o</literal>.
+ If parsing failed, yaz_pqf_scan returns a NULL pointer.
+ Error information for bad queries can be obtained by a call to
+ <function>yaz_pqf_error</function> which returns an error code and
+ modifies <literal>*msg</literal> to point to an error description,
+ and modifies <literal>*off</literal> to the offset within last
+ query were parsing failed.
+ </para>
+ <para>
+ The second set of functions are declared as follows:
+ </para>
+ <synopsis>
+ #include <yaz/pquery.h>
+
+ Z_RPNQuery *p_query_rpn(ODR o, oid_proto proto, const char *qbuf);
+
+ Z_AttributesPlusTerm *p_query_scan(ODR o, oid_proto proto,
+ Odr_oid **attributeSetP, const char *qbuf);
+
+ int p_query_attset(const char *arg);
+ </synopsis>
+ <para>
+ The function <function>p_query_rpn()</function> takes as arguments an
+ &odr; stream (see section <link linkend="odr">The ODR Module</link>)
+ to provide a memory source (the structure created is released on
+ the next call to <function>odr_reset()</function> on the stream), a
+ protocol identifier (one of the constants <token>PROTO_Z3950</token> and
+ <token>PROTO_SR</token>), an attribute set reference, and
+ finally a null-terminated string holding the query string.
+ </para>
+ <para>
+ If the parse went well, <function>p_query_rpn()</function> returns a
+ pointer to a <literal>Z_RPNQuery</literal> structure which can be
+ placed directly into a <literal>Z_SearchRequest</literal>.
+ If parsing failed, due to syntax error, a NULL pointer is returned.
+ </para>
+ <para>
+ The <literal>p_query_attset</literal> specifies which attribute set
+ to use if the query doesn't specify one by the
+ <literal>@attrset</literal> operator.
+ The <literal>p_query_attset</literal> returns 0 if the argument is a
+ valid attribute set specifier; otherwise the function returns -1.
+ </para>
+ <para>
+ The grammar of the PQF is as follows:
+ </para>
+ <literallayout>
+ query ::= top-set query-struct.
+
+ top-set ::= [ '@attrset' string ]
+
+ query-struct ::= attr-spec | simple | complex | '@term' term-type query
+
+ attr-spec ::= '@attr' [ string ] string query-struct
+
+ complex ::= operator query-struct query-struct.
+
+ operator ::= '@and' | '@or' | '@not' | '@prox' proximity.
+
+ simple ::= result-set | term.
+
+ result-set ::= '@set' string.
+
+ term ::= string.
+
+ proximity ::= exclusion distance ordered relation which-code unit-code.
+
+ exclusion ::= '1' | '0' | 'void'.
+
+ distance ::= integer.
+
+ ordered ::= '1' | '0'.
+
+ relation ::= integer.
+
+ which-code ::= 'known' | 'private' | integer.
+
+ unit-code ::= integer.
+
+ term-type ::= 'general' | 'numeric' | 'string' | 'oid' | 'datetime' | 'null'.
+ </literallayout>
+ <para>
+ You will note that the syntax above is a fairly faithful
+ representation of RPN, except for the Attribute, which has been
+ moved a step away from the term, allowing you to associate one or more
+ attributes with an entire query structure. The parser will
+ automatically apply the given attributes to each term as required.
+ </para>
+ <para>
+ The @attr operator is followed by an attribute specification
+ (<literal>attr-spec</literal> above). The specification consists
+ of an optional attribute set, an attribute type-value pair and
+ a sub-query. The attribute type-value pair is packed in one string:
+ an attribute type, an equals sign, and an attribute value, like this:
+ <literal>@attr 1=1003</literal>.
+ The type is always an integer but the value may be either an
+ integer or a string (if it doesn't start with a digit character).
+ A string attribute-value is encoded as a Type-1 ``complex''
+ attribute with the list of values containing the single string
+ specified, and including no semantic indicators.
+ </para>
+ <para>
+ Version 3 of the Z39.50 specification defines various encoding of terms.
+ Use <literal>@term </literal> <replaceable>type</replaceable>
+ <replaceable>string</replaceable>,
+ where type is one of: <literal>general</literal>,
+ <literal>numeric</literal> or <literal>string</literal>
+ (for InternationalString).
+ If no term type has been given, the <literal>general</literal> form
+ is used. This is the only encoding allowed in both versions 2 and 3
+ of the Z39.50 standard.
+ </para>
+ <sect3 id="PQF-prox">
+ <title>Using Proximity Operators with PQF</title>
+ <note>
+ <para>
+ This is an advanced topic, describing how to construct
+ queries that make very specific requirements on the
+ relative location of their operands.
+ You may wish to skip this section and go straight to
+ <link linkend="pqf-examples">the example PQF queries</link>.
+ </para>
+ <para>
+ <warning>
+ <para>
+ Most Z39.50 servers do not support proximity searching, or
+ support only a small subset of the full functionality that
+ can be expressed using the PQF proximity operator. Be
+ aware that the ability to <emphasis>express</emphasis> a
+ query in PQF is no guarantee that any given server will
+ be able to <emphasis>execute</emphasis> it.
+ </para>
+ </warning>
+ </para>
+ </note>
+ <para>
+ The proximity operator <literal>@prox</literal> is a special
+ and more restrictive version of the conjunction operator
+ <literal>@and</literal>. Its semantics are described in
+ section 3.7.2 (Proximity) of Z39.50 the standard itself, which
+ can be read on-line at
+ <ulink url="&url.z39.50.proximity;"/>
+ </para>
+ <para>
+ In PQF, the proximity operation is represented by a sequence
+ of the form
+ <screen>
+ @prox <replaceable>exclusion</replaceable> <replaceable>distance</replaceable> <replaceable>ordered</replaceable> <replaceable>relation</replaceable> <replaceable>which-code</replaceable> <replaceable>unit-code</replaceable>
+ </screen>
+ in which the meanings of the parameters are as described in in
+ the standard, and they can take the following values:
+ <itemizedlist>
+ <listitem>
+ <formalpara><title>exclusion</title>
+ <para>
+ 0 = false (i.e. the proximity condition specified by the
+ remaining parameters must be satisfied) or
+ 1 = true (the proximity condition specified by the
+ remaining parameters must <emphasis>not</emphasis> be
+ satisifed).
+ </para>
+ </formalpara>
+ </listitem>
+ <listitem>
+ <formalpara><title>distance</title><para>
+ An integer specifying the difference between the locations
+ of the operands: e.g. two adjacent words would have
+ distance=1 since their locations differ by one unit.
+ </para>
+ </formalpara></listitem>
+ <listitem>
+ <formalpara><title>ordered</title><para>
+ 1 = ordered (the operands must occur in the order the
+ query specifies them) or
+ 0 = unordered (they may appear in either order).
+ </para>
+ </formalpara>
+ </listitem>
+ <listitem>
+ <formalpara><title>relation</title><para>
+ Recognised values are
+ 1 (lessThan),
+ 2 (lessThanOrEqual),
+ 3 (equal),
+ 4 (greaterThanOrEqual),
+ 5 (greaterThan) and
+ 6 (notEqual).
+ </para>
+ </formalpara>
+ </listitem>
+ <listitem>
+ <formalpara><title>which-code</title><para>
+ <literal>known</literal>
+ or
+ <literal>k</literal>
+ (the unit-code parameter is taken from the well-known list
+ of alternatives described in below) or
+ <literal>private</literal>
+ or
+ <literal>p</literal>
+ (the unit-code paramater has semantics specific to an
+ out-of-band agreement such as a profile).
+ </para>
+ </formalpara>
+ </listitem>
+ <listitem>
+ <formalpara><title>unit-code</title><para>
+ If the which-code parameter is <literal>known</literal>
+ then the recognised values are
+ 1 (character),
+ 2 (word),
+ 3 (sentence),
+ 4 (paragraph),
+ 5 (section),
+ 6 (chapter),
+ 7 (document),
+ 8 (element),
+ 9 (subelement),
+ 10 (elementType) and
+ 11 (byte).
+ If which-code is <literal>private</literal> then the
+ acceptable values are determined by the profile.
+ </para>
+ </formalpara>
+ </listitem>
+ </itemizedlist>
+ (The numeric values of the relation and well-known unit-code
+ parameters are taken straight from
+ <ulink url="&url.z39.50.proximity.asn1;"
+ >the ASN.1</ulink> of the proximity structure in the standard.)
+ </para>
+ </sect3>
+ <sect3 id="pqf-examples">
+ <title>PQF queries</title>
+ <example id="example.pqf.simple.terms">
+ <title>PQF queries using simple terms</title>
+ <para>
+ <screen>
+ dylan
+
+ "bob dylan"
+ </screen>
+ </para>
+ </example>
+ <example id="pqf.example.pqf.boolean.operators">
+ <title>PQF boolean operators</title>
+ <para>
+ <screen>
+ @or "dylan" "zimmerman"
+
+ @and @or dylan zimmerman when
+
+ @and when @or dylan zimmerman
+ </screen>
+ </para>
+ </example>
+ <example id="example.pqf.result.sets">
+ <title>PQF references to result sets</title>
+ <para>
+ <screen>
+ @set Result-1
+
+ @and @set seta @set setb
+ </screen>
+ </para>
+ </example>
+ <example id="example.pqf.attributes">
+ <title>Attributes for terms</title>
+ <para>
+ <screen>
+ @attr 1=4 computer
+
+ @attr 1=4 @attr 4=1 "self portrait"
+
+ @attrset exp1 @attr 1=1 CategoryList
+
+ @attr gils 1=2008 Copenhagen
+
+ @attr 1=/book/title computer
+ </screen>
+ </para>
+ </example>
+ <example id="example.pqf.proximity">
+ <title>PQF Proximity queries</title>
+ <para>
+ <screen>
+ @prox 0 3 1 2 k 2 dylan zimmerman
+ </screen>
+ Here the parameters 0, 3, 1, 2, k and 2 represent exclusion,
+ distance, ordered, relation, which-code and unit-code, in that
+ order. So:
+ <itemizedlist>
+ <listitem>
+ <para>exclusion = 0: the proximity condition must hold</para>
+ </listitem>
+ <listitem>
+ <para>distance = 3: the terms must be three units apart</para>
+ </listitem>
+ <listitem>
+ <para>
+ ordered = 1: they must occur in the order they are specified
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ relation = 2: lessThanOrEqual (to the distance of 3 units)
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ which-code is ``known'', so the standard unit-codes are used
+ </para>
+ </listitem>
+ <listitem>
+ <para>unit-code = 2: word.</para>
+ </listitem>
+ </itemizedlist>
+ So the whole proximity query means that the words
+ <literal>dylan</literal> and <literal>zimmerman</literal> must
+ both occur in the record, in that order, differing in position
+ by three or fewer words (i.e. with two or fewer words between
+ them.) The query would find ``Bob Dylan, aka. Robert
+ Zimmerman'', but not ``Bob Dylan, born as Robert Zimmerman''
+ since the distance in this case is four.
+ </para>
+ </example>
+ <example id="example.pqf.search.term.type">
+ <title>PQF specification of search term type</title>
+ <para>
+ <screen>
+ @term string "a UTF-8 string, maybe?"
+ </screen>
+ </para>
+ </example>
+ <example id="example.pqf.mixed.queries">
+ <title>PQF mixed queries</title>
+ <para>
+ <screen>
+ @or @and bob dylan @set Result-1
+
+ @attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming"
+
+ @and @attr 2=4 @attr gils 1=2038 -114 @attr 2=2 @attr gils 1=2039 -109
+ </screen>
+ The last of these examples is a spatial search: in
+ <ulink url="http://www.gils.net/prof_v2.html#sec_7_4"
+ >the GILS attribute set</ulink>,
+ access point
+ 2038 indicates West Bounding Coordinate and
+ 2030 indicates East Bounding Coordinate,
+ so the query is for areas extending from -114 degrees
+ to no more than -109 degrees.
+ </para>
+ </example>
+ </sect3>
+ </sect2>
+ <sect2 id="CCL"><title>CCL</title>
+ <para>
+ Not all users enjoy typing in prefix query structures and numerical
+ attribute values, even in a minimalistic test client. In the library
+ world, the more intuitive Common Command Language - CCL (ISO 8777)
+ has enjoyed some popularity - especially before the widespread
+ availability of graphical interfaces. It is still useful in
+ applications where you for some reason or other need to provide a
+ symbolic language for expressing boolean query structures.
+ </para>
+ <sect3 id="ccl.syntax">
+ <title>CCL Syntax</title>
+ <para>
+ The CCL parser obeys the following grammar for the FIND argument.
+ The syntax is annotated by in the lines prefixed by
+ <literal>--</literal>.
+ </para>
+ <screen>
+ CCL-Find ::= CCL-Find Op Elements
+ | Elements.
+
+ Op ::= "and" | "or" | "not"
+ -- The above means that Elements are separated by boolean operators.
+
+ Elements ::= '(' CCL-Find ')'
+ | Set
+ | Terms
+ | Qualifiers Relation Terms
+ | Qualifiers Relation '(' CCL-Find ')'
+ | Qualifiers '=' string '-' string
+ -- Elements is either a recursive definition, a result set reference, a
+ -- list of terms, qualifiers followed by terms, qualifiers followed
+ -- by a recursive definition or qualifiers in a range (lower - upper).
+
+ Set ::= 'set' = string
+ -- Reference to a result set
+
+ Terms ::= Terms Prox Term
+ | Term
+ -- Proximity of terms.
+
+ Term ::= Term string
+ | string
+ -- This basically means that a term may include a blank
+
+ Qualifiers ::= Qualifiers ',' string
+ | string
+ -- Qualifiers is a list of strings separated by comma
+
+ Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<'
+ -- Relational operators. This really doesn't follow the ISO8777
+ -- standard.
+
+ Prox ::= '%' | '!'
+ -- Proximity operator
+
+ </screen>
+ <example id="example.ccl.queries">
+ <title>CCL queries</title>
+ <para>
+ The following queries are all valid:
+ </para>
+ <screen>
+ dylan
+
+ "bob dylan"
+
+ dylan or zimmerman
+
+ set=1
+
+ (dylan and bob) or set=1
+
+ righttrunc?
+
+ "notrunc?"
+
+ singlechar#mask
+ </screen>
+ <para>
+ Assuming that the qualifiers <literal>ti</literal>,
+ <literal>au</literal>
+ and <literal>date</literal> are defined we may use:
+ </para>
+ <screen>
+ ti=self portrait
+
+ au=(bob dylan and slow train coming)
+
+ date>1980 and (ti=((self portrait)))
+ </screen>
+ </example>
+ </sect3>
+ <sect3 id="ccl.qualifiers">
+ <title>CCL Qualifiers</title>
+ <para>
+ Qualifiers are used to direct the search to a particular searchable
+ index, such as title (ti) and author indexes (au). The CCL standard
+ itself doesn't specify a particular set of qualifiers, but it does
+ suggest a few short-hand notations. You can customize the CCL parser
+ to support a particular set of qualifiers to reflect the current target
+ profile. Traditionally, a qualifier would map to a particular
+ use-attribute within the BIB-1 attribute set. It is also
+ possible to set other attributes, such as the structure
+ attribute.
+ </para>
+ <para>
+ A CCL profile is a set of predefined CCL qualifiers that may be
+ read from a file or set in the CCL API.
+ The YAZ client reads its CCL qualifiers from a file named
+ <filename>default.bib</filename>. There are four types of
+ lines in a CCL profile: qualifier specification,
+ qualifier alias, comments and directives.
+ </para>
+ <sect4 id="ccl.qualifier.specification">
+ <title>Qualifier specification</title>
+ <para>
+ A qualifier specification is of the form:
+ </para>
+ <para>
+ <replaceable>qualifier-name</replaceable>
+ [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable>
+ [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable> ...
+ </para>
+ <para>
+ where <replaceable>qualifier-name</replaceable> is the name of the
+ qualifier to be used (eg. <literal>ti</literal>),
+ <replaceable>type</replaceable> is attribute type in the attribute
+ set (Bib-1 is used if no attribute set is given) and
+ <replaceable>val</replaceable> is attribute value.
+ The <replaceable>type</replaceable> can be specified as an
+ integer or as it be specified either as a single-letter:
+ <literal>u</literal> for use,
+ <literal>r</literal> for relation,<literal>p</literal> for position,
+ <literal>s</literal> for structure,<literal>t</literal> for truncation
+ or <literal>c</literal> for completeness.
+ The attributes for the special qualifier name <literal>term</literal>
+ are used when no CCL qualifier is given in a query.
+ <table id="ccl.common.bib1.attributes">
+ <title>Common Bib-1 attributes</title>
+ <tgroup cols="2">
+ <colspec colwidth="2*" colname="type"></colspec>
+ <colspec colwidth="9*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>u=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Use attribute (1). Common use attributes are
+ 1 Personal-name, 4 Title, 7 ISBN, 8 ISSN, 30 Date,
+ 62 Subject, 1003 Author), 1016 Any. Specify value
+ as an integer.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>r=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Relation attribute (2). Common values are
+ 1 <, 2 <=, 3 =, 4 >=, 5 >, 6 <>,
+ 100 phonetic, 101 stem, 102 relevance, 103 always matches.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>p=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Position attribute (3). Values: 1 first in field, 2
+ first in any subfield, 3 any position in field.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>s=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Structure attribute (4). Values: 1 phrase, 2 word,
+ 3 key, 4 year, 5 date, 6 word list, 100 date (un),
+ 101 name (norm), 102 name (un), 103 structure, 104 urx,
+ 105 free-form-text, 106 document-text, 107 local-number,
+ 108 string, 109 numeric string.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>t=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Truncation attribute (5). Values: 1 right, 2 left,
+ 3 left& right, 100 none, 101 process #, 102 regular-1,
+ 103 regular-2, 104 CCL.
+ </entry>
+ </row>
+ <row>
+ <entry><literal>c=</literal><replaceable>value</replaceable></entry>
+ <entry>
+ Completeness attribute (6). Values: 1 incomplete subfield,
+ 2 complete subfield, 3 complete field.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </para>
+ <para>
+ Refer to <xref linkend="bib1"/> or the complete
+ <ulink url="&url.z39.50.attset.bib1;">list of Bib-1 attributes</ulink>
+ </para>
+ <para>
+ It is also possible to specify non-numeric attribute values,
+ which are used in combination with certain types.
+ The special combinations are:
+ <table id="ccl.special.attribute.combos">
+ <title>Special attribute combos</title>
+ <tgroup cols="2">
+ <colspec colwidth="2*" colname="name"></colspec>
+ <colspec colwidth="9*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>s=pw</literal></entry>
+ <entry>
+ The structure is set to either word or phrase depending
+ on the number of tokens in a term (phrase-word).
+ </entry>
+ </row>
+ <row>
+ <entry><literal>s=al</literal></entry>
+ <entry>
+ Each token in the term is ANDed. (and-list).
+ This does not set the structure at all.
+ </entry>
+ </row>
+ <row><entry><literal>s=ol</literal></entry>
+ <entry>
+ Each token in the term is ORed. (or-list).
+ This does not set the structure at all.
+ </entry>
+ </row>
+ <row><entry><literal>s=ag</literal></entry>
+ <entry>
+ Tokens that appears as phrases (with blank in them) gets
+ structure phrase attached (4=1). Tokens that appear to be words
+ gets structure word attached (4=2). Phrases and words are
+ ANDed. This is a variant of s=al and s=pw, with the main
+ difference that words are not split (with operator AND)
+ but instead kept in one RPN token. This facility appeared
+ in YAZ 4.2.38.
+ </entry>
+ </row>
+ <row><entry><literal>r=o</literal></entry>
+ <entry>
+ Allows ranges and the operators greather-than, less-than, ...
+ equals.
+ This sets Bib-1 relation attribute accordingly (relation
+ ordered). A query construct is only treated as a range if
+ dash is used and that is surrounded by white-space. So
+ <literal>-1980</literal> is treated as term
+ <literal>"-1980"</literal> not <literal><= 1980</literal>.
+ If <literal>- 1980</literal> is used, however, that is
+ treated as a range.
+ </entry>
+ </row>
+ <row><entry><literal>r=r</literal></entry>
+ <entry>
+ Similar to <literal>r=o</literal> but assumes that terms
+ are non-negative (not prefixed with <literal>-</literal>).
+ Thus, a dash will always be treated as a range.
+ The construct <literal>1980-1990</literal> is
+ treated as a range with <literal>r=r</literal> but as a
+ single term <literal>"1980-1990"</literal> with
+ <literal>r=o</literal>. The special attribute
+ <literal>r=r</literal> is available in YAZ 2.0.24 or later.
+ </entry>
+ </row>
+ <row><entry><literal>t=l</literal></entry>
+ <entry>
+ Allows term to be left-truncated.
+ If term is of the form <literal>?x</literal>, the resulting
+ Type-1 term is <literal>x</literal> and truncation is left.
+ </entry>
+ </row>
+ <row><entry><literal>t=r</literal></entry>
+ <entry>
+ Allows term to be right-truncated.
+ If term is of the form <literal>x?</literal>, the resulting
+ Type-1 term is <literal>x</literal> and truncation is right.
+ </entry>
+ </row>
+ <row><entry><literal>t=n</literal></entry>
+ <entry>
+ If term is does not include <literal>?</literal>, the
+ truncation attribute is set to none (100).
+ </entry>
+ </row>
+ <row><entry><literal>t=b</literal></entry>
+ <entry>
+ Allows term to be both left&right truncated.
+ If term is of the form <literal>?x?</literal>, the
+ resulting term is <literal>x</literal> and trunctation is
+ set to both left&right.
+ </entry>
+ </row>
+ <row><entry><literal>t=x</literal></entry>
+ <entry>
+ Allows masking anywhere in a term, thus fully supporting
+ # (mask one character) and ? (zero or more of any).
+ If masking is used, trunction is set to 102 (regexp-1 in term)
+ and the term is converted accordingly to a regular expression.
+ </entry>
+ </row>
+ <row><entry><literal>t=z</literal></entry>
+ <entry>
+ Allows masking anywhere in a term, thus fully supporting
+ # (mask one character) and ? (zero or more of any).
+ If masking is used, trunction is set to 104 (Z39.58 in term)
+ and the term is converted accordingly to Z39.58 masking term -
+ actually the same truncation as CCL itself.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </para>
+ <example id="example.ccl.profile">
+ <title>CCL profile</title>
+ <para>
+ Consider the following definition:
+ </para>
+ <screen>
+ ti u=4 s=1
+ au u=1 s=1
+ term s=105
+ ranked r=102
+ date u=30 r=o
+ </screen>
+ <para>
+ <literal>ti</literal> and <literal>au</literal> both set
+ structure attribute to phrase (s=1).
+ <literal>ti</literal>
+ sets the use-attribute to 4. <literal>au</literal> sets the
+ use-attribute to 1.
+ When no qualifiers are used in the query the structure-attribute is
+ set to free-form-text (105) (rule for <literal>term</literal>).
+ The <literal>date</literal> sets the relation attribute to
+ the relation used in the CCL query and sets the use attribute
+ to 30 (Bib-1 Date).
+ </para>
+ <para>
+ You can combine attributes. To Search for "ranked title" you
+ can do
+ <screen>
+ ti,ranked=knuth computer
+ </screen>
+ which will set relation=ranked, use=title, structure=phrase.
+ </para>
+ <para>
+ Query
+ <screen>
+ date > 1980
+ </screen>
+ is a valid query. But
+ <screen>
+ ti > 1980
+ </screen>
+ is invalid.
+ </para>
+ </example>
+ </sect4>
+ <sect4 id="ccl.qualifier.alias">
+ <title>Qualifier alias</title>
+ <para>
+ A qualifier alias is of the form:
+ </para>
+ <para>
+ <replaceable>q</replaceable>
+ <replaceable>q1</replaceable> <replaceable>q2</replaceable> ..
+ </para>
+ <para>
+ which declares <replaceable>q</replaceable> to
+ be an alias for <replaceable>q1</replaceable>,
+ <replaceable>q2</replaceable>... such that the CCL
+ query <replaceable>q=x</replaceable> is equivalent to
+ <replaceable>q1=x or q2=x or ...</replaceable>.
+ </para>
+ </sect4>
+ <sect4 id="ccl.comments">
+ <title>Comments</title>
+ <para>
+ Lines with white space or lines that begin with
+ character <literal>#</literal> are treated as comments.
+ </para>
+ </sect4>
+ <sect4 id="ccl.directives">
+ <title>Directives</title>
+ <para>
+ Directive specifications takes the form
+ </para>
+ <para><literal>@</literal><replaceable>directive</replaceable> <replaceable>value</replaceable>
+ </para>
+ <table id="ccl.directives.table">
+ <title>CCL directives</title>
+ <tgroup cols="3">
+ <colspec colwidth="2*" colname="name"></colspec>
+ <colspec colwidth="8*" colname="description"></colspec>
+ <colspec colwidth="1*" colname="default"></colspec>
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Description</entry>
+ <entry>Default</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>truncation</entry>
+ <entry>Truncation character</entry>
+ <entry><literal>?</literal></entry>
+ </row>
+ <row>
+ <entry>mask</entry>
+ <entry>Masking character. Requires YAZ 4.2.58 or later</entry>
+ <entry><literal>#</literal></entry>
+ </row>
+ <row>
+ <entry>field</entry>
+ <entry>Specifies how multiple fields are to be
+ combined. There are two modes: <literal>or</literal>:
+ multiple qualifier fields are ORed,
+ <literal>merge</literal>: attributes for the qualifier
+ fields are merged and assigned to one term.
+ </entry>
+ <entry><literal>merge</literal></entry>
+ </row>
+ <row>
+ <entry>case</entry>
+ <entry>Specifies if CCL operators and qualifiers should be
+ compared with case sensitivity or not. Specify 1 for
+ case sensitive; 0 for case insensitive.</entry>
+ <entry><literal>1</literal></entry>
+ </row>
+ <row>
+ <entry>and</entry>
+ <entry>Specifies token for CCL operator AND.</entry>
+ <entry><literal>and</literal></entry>
+ </row>
+ <row>
+ <entry>or</entry>
+ <entry>Specifies token for CCL operator OR.</entry>
+ <entry><literal>or</literal></entry>
+ </row>
+ <row>
+ <entry>not</entry>
+ <entry>Specifies token for CCL operator NOT.</entry>
+ <entry><literal>not</literal></entry>
+ </row>
+ <row>
+ <entry>set</entry>
+ <entry>Specifies token for CCL operator SET.</entry>
+ <entry><literal>set</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect4>
+ </sect3>
+ <sect3 id="ccl.api">
+ <title>CCL API</title>
+ <para>
+ All public definitions can be found in the header file
+ <filename>ccl.h</filename>. A profile identifier is of type
+ <literal>CCL_bibset</literal>. A profile must be created with the call
+ to the function <function>ccl_qual_mk</function> which returns a profile
+ handle of type <literal>CCL_bibset</literal>.
+ </para>
+ <para>
+ To read a file containing qualifier definitions the function
+ <function>ccl_qual_file</function> may be convenient. This function
+ takes an already opened <literal>FILE</literal> handle pointer as
+ argument along with a <literal>CCL_bibset</literal> handle.
+ </para>
+ <para>
+ To parse a simple string with a FIND query use the function
+ </para>
+ <screen>
+struct ccl_rpn_node *ccl_find_str(CCL_bibset bibset, const char *str,
+ int *error, int *pos);
+ </screen>
+ <para>
+ which takes the CCL profile (<literal>bibset</literal>) and query
+ (<literal>str</literal>) as input. Upon successful completion the RPN
+ tree is returned. If an error occur, such as a syntax error, the integer
+ pointed to by <literal>error</literal> holds the error code and
+ <literal>pos</literal> holds the offset inside query string in which
+ the parsing failed.
+ </para>
+ <para>
+ An English representation of the error may be obtained by calling
+ the <literal>ccl_err_msg</literal> function. The error codes are
+ listed in <filename>ccl.h</filename>.
+ </para>
+ <para>
+ To convert the CCL RPN tree (type
+ <literal>struct ccl_rpn_node *</literal>)
+ to the Z_RPNQuery of YAZ the function <function>ccl_rpn_query</function>
+ must be used. This function which is part of YAZ is implemented in
+ <filename>yaz-ccl.c</filename>.
+ After calling this function the CCL RPN tree is probably no longer
+ needed. The <literal>ccl_rpn_delete</literal> destroys the CCL RPN tree.
+ </para>
+ <para>
+ A CCL profile may be destroyed by calling the
+ <function>ccl_qual_rm</function> function.
+ </para>
+ <para>
+ The token names for the CCL operators may be changed by setting the
+ globals (all type <literal>char *</literal>)
+ <literal>ccl_token_and</literal>, <literal>ccl_token_or</literal>,
+ <literal>ccl_token_not</literal> and <literal>ccl_token_set</literal>.
+ An operator may have aliases, i.e. there may be more than one name for
+ the operator. To do this, separate each alias with a space character.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2 id="cql">
+ <title>CQL</title>
+ <para>
+ <ulink url="&url.cql;">CQL</ulink>
+ - Common Query Language - was defined for the
+ <ulink url="&url.sru;">SRU</ulink> protocol.
+ In many ways CQL has a similar syntax to CCL.
+ The objective of CQL is different. Where CCL aims to be
+ an end-user language, CQL is <emphasis>the</emphasis> protocol
+ query language for SRU.
+ </para>
+ <tip>
+ <para>
+ If you are new to CQL, read the
+ <ulink url="&url.cql.intro;">Gentle Introduction</ulink>.
+ </para>
+ </tip>
+ <para>
+ The CQL parser in &yaz; provides the following:
+ <itemizedlist>
+ <listitem>
+ <para>
+ It parses and validates a CQL query.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ It generates a C structure that allows you to convert
+ a CQL query to some other query language, such as SQL.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The parser converts a valid CQL query to PQF, thus providing a
+ way to use CQL for both SRU servers and Z39.50 targets at the
+ same time.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The parser converts CQL to XCQL.
+ XCQL is an XML representation of CQL.
+ XCQL is part of the SRU specification. However, since SRU
+ supports CQL only, we don't expect XCQL to be widely used.
+ Furthermore, CQL has the advantage over XCQL that it is
+ easy to read.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <sect3 id="cql.parsing">
+ <title>CQL parsing</title>
+ <para>
+ A CQL parser is represented by the <literal>CQL_parser</literal>
+ handle. Its contents should be considered &yaz; internal (private).
+ <synopsis>
+#include <yaz/cql.h>
+
+typedef struct cql_parser *CQL_parser;
+
+CQL_parser cql_parser_create(void);
+void cql_parser_destroy(CQL_parser cp);
+ </synopsis>
+ A parser is created by <function>cql_parser_create</function> and
+ is destroyed by <function>cql_parser_destroy</function>.
+ </para>
+ <para>
+ To parse a CQL query string, the following function
+ is provided:
+ <synopsis>
+int cql_parser_string(CQL_parser cp, const char *str);
+ </synopsis>
+ A CQL query is parsed by the <function>cql_parser_string</function>
+ which takes a query <parameter>str</parameter>.
+ If the query was valid (no syntax errors), then zero is returned;
+ otherwise -1 is returned to indicate a syntax error.
+ </para>
+ <para>
+ <synopsis>
+int cql_parser_stream(CQL_parser cp,
+ int (*getbyte)(void *client_data),
+ void (*ungetbyte)(int b, void *client_data),
+ void *client_data);
+
+int cql_parser_stdio(CQL_parser cp, FILE *f);
+ </synopsis>
+ The functions <function>cql_parser_stream</function> and
+ <function>cql_parser_stdio</function> parses a CQL query
+ - just like <function>cql_parser_string</function>.
+ The only difference is that the CQL query can be
+ fed to the parser in different ways.
+ The <function>cql_parser_stream</function> uses a generic
+ byte stream as input. The <function>cql_parser_stdio</function>
+ uses a <literal>FILE</literal> handle which is opened for reading.
+ </para>
+ </sect3>
+ <sect3 id="cql.tree">
+ <title>CQL tree</title>
+ <para>
+ The the query string is valid, the CQL parser
+ generates a tree representing the structure of the
+ CQL query.
+ </para>
+ <para>
+ <synopsis>
+struct cql_node *cql_parser_result(CQL_parser cp);
+ </synopsis>
+ <function>cql_parser_result</function> returns the
+ a pointer to the root node of the resulting tree.
+ </para>
+ <para>
+ Each node in a CQL tree is represented by a
+ <literal>struct cql_node</literal>.
+ It is defined as follows:
+ <synopsis>
+#define CQL_NODE_ST 1
+#define CQL_NODE_BOOL 2
+#define CQL_NODE_SORT 3
+struct cql_node {
+ int which;
+ union {
+ struct {
+ char *index;
+ char *index_uri;
+ char *term;
+ char *relation;
+ char *relation_uri;
+ struct cql_node *modifiers;
+ } st;
+ struct {
+ char *value;
+ struct cql_node *left;
+ struct cql_node *right;
+ struct cql_node *modifiers;
+ } boolean;
+ struct {
+ char *index;
+ struct cql_node *next;
+ struct cql_node *modifiers;
+ struct cql_node *search;
+ } sort;
+ } u;
+};
+ </synopsis>
+ There are three node types: search term (ST), boolean (BOOL)
+ and sortby (SORT).
+ A modifier is treated as a search term too.
+ </para>
+ <para>
+ The search term node has five members:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>index</literal>: index for search term.
+ If an index is unspecified for a search term,
+ <literal>index</literal> will be NULL.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>index_uri</literal>: index URi for search term
+ or NULL if none could be resolved for the index.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>term</literal>: the search term itself.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>relation</literal>: relation for search term.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>relation_uri</literal>: relation URI for search term.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>modifiers</literal>: relation modifiers for search
+ term. The <literal>modifiers</literal> list itself of cql_nodes
+ each of type <literal>ST</literal>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The boolean node represents <literal>and</literal>,
+ <literal>or</literal>, <literal>not</literal> +
+ proximity.
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>left</literal> and <literal>right</literal>: left
+ - and right operand respectively.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>modifiers</literal>: proximity arguments.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The sort node represents both the SORTBY clause.
+ </para>
+ </sect3>
+ <sect3 id="cql.to.pqf">
+ <title>CQL to PQF conversion</title>
+ <para>
+ Conversion to PQF (and Z39.50 RPN) is tricky by the fact
+ that the resulting RPN depends on the Z39.50 target
+ capabilities (combinations of supported attributes).
+ In addition, the CQL and SRU operates on index prefixes
+ (URI or strings), whereas the RPN uses Object Identifiers
+ for attribute sets.
+ </para>
+ <para>
+ The CQL library of &yaz; defines a <literal>cql_transform_t</literal>
+ type. It represents a particular mapping between CQL and RPN.
+ This handle is created and destroyed by the functions:
+ <synopsis>
+cql_transform_t cql_transform_open_FILE (FILE *f);
+cql_transform_t cql_transform_open_fname(const char *fname);
+void cql_transform_close(cql_transform_t ct);
+ </synopsis>
+ The first two functions create a tranformation handle from
+ either an already open FILE or from a filename respectively.
+ </para>
+ <para>
+ The handle is destroyed by <function>cql_transform_close</function>
+ in which case no further reference of the handle is allowed.
+ </para>
+ <para>
+ When a <literal>cql_transform_t</literal> handle has been created
+ you can convert to RPN.
+ <synopsis>
+int cql_transform_buf(cql_transform_t ct,
+ struct cql_node *cn, char *out, int max);
+ </synopsis>
+ This function converts the CQL tree <literal>cn</literal>
+ using handle <literal>ct</literal>.
+ For the resulting PQF, you supply a buffer <literal>out</literal>
+ which must be able to hold at at least <literal>max</literal>
+ characters.
+ </para>
+ <para>
+ If conversion failed, <function>cql_transform_buf</function>
+ returns a non-zero SRU error code; otherwise zero is returned
+ (conversion successful). The meanings of the numeric error
+ codes are listed in the SRU specification somewhere (no
+ direct link anymore).
+ </para>
+ <para>
+ If conversion fails, more information can be obtained by calling
+ <synopsis>
+int cql_transform_error(cql_transform_t ct, char **addinfop);
+ </synopsis>
+ This function returns the most recently returned numeric
+ error-code and sets the string-pointer at
+ <literal>*addinfop</literal> to point to a string containing
+ additional information about the error that occurred: for
+ example, if the error code is 15 (``Illegal or unsupported context
+ set''), the additional information is the name of the requested
+ context set that was not recognised.
+ </para>
+ <para>
+ The SRU error-codes may be translated into brief human-readable
+ error messages using
+ <synopsis>
+const char *cql_strerror(int code);
+ </synopsis>
+ </para>
+ <para>
+ If you wish to be able to produce a PQF result in a different
+ way, there are two alternatives.
+ <synopsis>
+void cql_transform_pr(cql_transform_t ct,
+ struct cql_node *cn,
+ void (*pr)(const char *buf, void *client_data),
+ void *client_data);
+
+int cql_transform_FILE(cql_transform_t ct,
+ struct cql_node *cn, FILE *f);
+ </synopsis>
+ The former function produces output to a user-defined
+ output stream. The latter writes the result to an already
+ open <literal>FILE</literal>.
+ </para>
+ </sect3>
+ <sect3 id="cql.to.rpn">
+ <title>Specification of CQL to RPN mappings</title>
+ <para>
+ The file supplied to functions
+ <function>cql_transform_open_FILE</function>,
+ <function>cql_transform_open_fname</function> follows
+ a structure found in many Unix utilities.
+ It consists of mapping specifications - one per line.
+ Lines starting with <literal>#</literal> are ignored (comments).
+ </para>
+ <para>
+ Each line is of the form
+ <literallayout>
+ <replaceable>CQL pattern</replaceable><literal> = </literal> <replaceable> RPN equivalent</replaceable>
+ </literallayout>
+ </para>
+ <para>
+ An RPN pattern is a simple attribute list. Each attribute pair
+ takes the form:
+ <literallayout>
+ [<replaceable>set</replaceable>] <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>
+ </literallayout>
+ The attribute <replaceable>set</replaceable> is optional.
+ The <replaceable>type</replaceable> is the attribute type,
+ <replaceable>value</replaceable> the attribute value.
+ </para>
+ <para>
+ The character <literal>*</literal> (asterisk) has special meaning
+ when used in the RPN pattern.
+ Each occurrence of <literal>*</literal> is substituted with the
+ CQL matching name (index, relation, qualifier etc).
+ This facility can be used to copy a CQL name verbatim to the RPN result.
+ </para>
+ <para>
+ The following CQL patterns are recognized:
+ <variablelist>
+ <varlistentry>
+ <term>
+ <literal>index.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern is invoked when a CQL index, such as
+ dc.title is converted. <replaceable>set</replaceable>
+ and <replaceable>name</replaceable> are the context set and index
+ name respectively.
+ Typically, the RPN specifies an equivalent use attribute.
+ </para>
+ <para>
+ For terms not bound by an index the pattern
+ <literal>index.cql.serverChoice</literal> is used.
+ Here, the prefix <literal>cql</literal> is defined as
+ <literal>http://www.loc.gov/zing/cql/cql-indexes/v1.0/</literal>.
+ If this pattern is not defined, the mapping will fail.
+ </para>
+ <para>
+ The pattern,
+ <literal>index.</literal><replaceable>set</replaceable><literal>.*</literal>
+ is used when no other index pattern is matched.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>qualifier.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
+ (DEPRECATED)
+ </term>
+ <listitem>
+ <para>
+ For backwards compatibility, this is recognised as a synonym of
+ <literal>index.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>relation.</literal><replaceable>relation</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern specifies how a CQL relation is mapped to RPN.
+ <replaceable>pattern</replaceable> is name of relation
+ operator. Since <literal>=</literal> is used as
+ separator between CQL pattern and RPN, CQL relations
+ including <literal>=</literal> cannot be
+ used directly. To avoid a conflict, the names
+ <literal>ge</literal>,
+ <literal>eq</literal>,
+ <literal>le</literal>,
+ must be used for CQL operators, greater-than-or-equal,
+ equal, less-than-or-equal respectively.
+ The RPN pattern is supposed to include a relation attribute.
+ </para>
+ <para>
+ For terms not bound by a relation, the pattern
+ <literal>relation.scr</literal> is used. If the pattern
+ is not defined, the mapping will fail.
+ </para>
+ <para>
+ The special pattern, <literal>relation.*</literal> is used
+ when no other relation pattern is matched.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>relationModifier.</literal><replaceable>mod</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern specifies how a CQL relation modifier is mapped to RPN.
+ The RPN pattern is usually a relation attribute.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>structure.</literal><replaceable>type</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern specifies how a CQL structure is mapped to RPN.
+ Note that this CQL pattern is somewhat to similar to
+ CQL pattern <literal>relation</literal>.
+ The <replaceable>type</replaceable> is a CQL relation.
+ </para>
+ <para>
+ The pattern, <literal>structure.*</literal> is used
+ when no other structure pattern is matched.
+ Usually, the RPN equivalent specifies a structure attribute.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>position.</literal><replaceable>type</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This pattern specifies how the anchor (position) of
+ CQL is mapped to RPN.
+ The <replaceable>type</replaceable> is one
+ of <literal>first</literal>, <literal>any</literal>,
+ <literal>last</literal>, <literal>firstAndLast</literal>.
+ </para>
+ <para>
+ The pattern, <literal>position.*</literal> is used
+ when no other position pattern is matched.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>set.</literal><replaceable>prefix</replaceable>
+ </term>
+ <listitem>
+ <para>
+ This specification defines a CQL context set for a given prefix.
+ The value on the right hand side is the URI for the set -
+ <emphasis>not</emphasis> RPN. All prefixes used in
+ index patterns must be defined this way.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>
+ <literal>set</literal>
+ </term>
+ <listitem>
+ <para>
+ This specification defines a default CQL context set for index names.
+ The value on the right hand side is the URI for the set.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <example id="example.cql.to.rpn.mapping">
+ <title>CQL to RPN mapping file</title>
+ <para>
+ This simple file defines two context sets, three indexes and three
+ relations, a position pattern and a default structure.
+ </para>
+ <programlisting><![CDATA[
+ set.cql = http://www.loc.gov/zing/cql/context-sets/cql/v1.1/
+ set.dc = http://www.loc.gov/zing/cql/dc-indexes/v1.0/
+
+ index.cql.serverChoice = 1=1016
+ index.dc.title = 1=4
+ index.dc.subject = 1=21
+
+ relation.< = 2=1
+ relation.eq = 2=3
+ relation.scr = 2=3
+
+ position.any = 3=3 6=1
+
+ structure.* = 4=1
+]]>
+ </programlisting>
+ <para>
+ With the mappings above, the CQL query
+ <screen>
+ computer
+ </screen>
+ is converted to the PQF:
+ <screen>
+ @attr 1=1016 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "computer"
+ </screen>
+ by rules <literal>index.cql.serverChoice</literal>,
+ <literal>relation.scr</literal>, <literal>structure.*</literal>,
+ <literal>position.any</literal>.
+ </para>
+ <para>
+ CQL query
+ <screen>
+ computer^
+ </screen>
+ is rejected, since <literal>position.right</literal> is
+ undefined.
+ </para>
+ <para>
+ CQL query
+ <screen>
+ >my = "http://www.loc.gov/zing/cql/dc-indexes/v1.0/" my.title = x
+ </screen>
+ is converted to
+ <screen>
+ @attr 1=4 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "x"
+ </screen>
+ </para>
+ </example>
+ <example id="example.cql.to.rpn.string">
+ <title>CQL to RPN string attributes</title>
+ <para>
+ In this example we allow any index to be passed to RPN as
+ a use attribute.
+ </para>
+ <programlisting><![CDATA[
+ # Identifiers for prefixes used in this file. (index.*)
+ set.cql = info:srw/cql-context-set/1/cql-v1.1
+ set.rpn = http://bogus/rpn
+ set = http://bogus/rpn
+
+ # The default index when none is specified by the query
+ index.cql.serverChoice = 1=any
+
+ index.rpn.* = 1=*
+ relation.eq = 2=3
+ structure.* = 4=1
+ position.any = 3=3
+]]>
+ </programlisting>
+ <para>
+ The <literal>http://bogus/rpn</literal> context set is also the default
+ so we can make queries such as
+ <screen>
+ title = a
+ </screen>
+ which is converted to
+ <screen>
+ @attr 2=3 @attr 4=1 @attr 3=3 @attr 1=title "a"
+ </screen>
+ </para>
+ </example>
+ <example id="example.cql.to.rpn.bathprofile">
+ <title>CQL to RPN using Bath Profile</title>
+ <para>
+ The file <filename>etc/pqf.properties</filename> has mappings from
+ the Bath Profile and Dublin Core to RPN.
+ If YAZ is installed as a package it's usually located
+ in <filename>/usr/share/yaz/etc</filename> and part of the
+ development package, such as <literal>libyaz-dev</literal>.
+ </para>
+ </example>
+ </sect3>
+ <sect3 id="cql.xcql">
+ <title>CQL to XCQL conversion</title>
+ <para>
+ Conversion from CQL to XCQL is trivial and does not
+ require a mapping to be defined.
+ There three functions to choose from depending on the
+ way you wish to store the resulting output (XML buffer
+ containing XCQL).
+ <synopsis>
+int cql_to_xml_buf(struct cql_node *cn, char *out, int max);
+void cql_to_xml(struct cql_node *cn,
+ void (*pr)(const char *buf, void *client_data),
+ void *client_data);
+void cql_to_xml_stdio(struct cql_node *cn, FILE *f);
+ </synopsis>
+ Function <function>cql_to_xml_buf</function> converts
+ to XCQL and stores result in a user supplied buffer of a given
+ max size.
+ </para>
+ <para>
+ <function>cql_to_xml</function> writes the result in
+ a user defined output stream.
+ <function>cql_to_xml_stdio</function> writes to a
+ a file.
+ </para>
+ </sect3>
+ <sect3 id="rpn.to.cql">
+ <title>PQF to CQL conversion</title>
+ <para>
+ Conversion from PQF to CQL is offered by the two functions shown
+ below. The former uses a generic stream for result. The latter
+ puts result in a WRBUF (string container).
+ <synopsis>
+#include <yaz/rpn2cql.h>
+
+int cql_transform_rpn2cql_stream(cql_transform_t ct,
+ void (*pr)(const char *buf, void *client_data),
+ void *client_data,
+ Z_RPNQuery *q);
+
+int cql_transform_rpn2cql_wrbuf(cql_transform_t ct,
+ WRBUF w,
+ Z_RPNQuery *q);
+ </synopsis>
+ The configuration is the same as used in CQL to PQF conversions.
+ </para>
+ </sect3>
+ </sect2>
+ </sect1>
+ <sect1 id="tools.oid">
+ <title>Object Identifiers</title>
+ <para>
+ The basic YAZ representation of an OID is an array of integers,
+ terminated with the value -1. This integer is of type
+ <literal>Odr_oid</literal>.
+ </para>
+ <para>
+ Fundamental OID operations and the type <literal>Odr_oid</literal>
+ are defined in <filename>yaz/oid_util.h</filename>.
+ </para>
+ <para>
+ An OID can either be declared as a automatic variable or it can
+ allocated using the memory utilities or ODR/NMEM. It's
+ guaranteed that an OID can fit in <literal>OID_SIZE</literal> integers.
+ </para>
+ <example id="tools.oid.bib1.1"><title>Create OID on stack</title>
+ <para>
+ We can create an OID for the Bib-1 attribute set with:
+ <screen>
+ Odr_oid bib1[OID_SIZE];
+ bib1[0] = 1;
+ bib1[1] = 2;
+ bib1[2] = 840;
+ bib1[3] = 10003;
+ bib1[4] = 3;
+ bib1[5] = 1;
+ bib1[6] = -1;
+ </screen>
+ </para>
+ </example>
+ <para>
+ And OID may also be filled from a string-based representation using
+ dots (.). This is achieved by function
+ <screen>
+ int oid_dotstring_to_oid(const char *name, Odr_oid *oid);
+ </screen>
+ This functions returns 0 if name could be converted; -1 otherwise.
+ </para>
+ <example id="tools.oid.bib1.2"><title>Using oid_oiddotstring_to_oid</title>
+ <para>
+ We can fill the Bib-1 attribute set OID easier with:
+ <screen>
+ Odr_oid bib1[OID_SIZE];
+ oid_oiddotstring_to_oid("1.2.840.10003.3.1", bib1);
+ </screen>
+ </para>
+ </example>
+ <para>
+ We can also allocate an OID dynamically on a ODR stream with:
+ <screen>
+ Odr_oid *odr_getoidbystr(ODR o, const char *str);
+ </screen>
+ This creates an OID from string-based representation using dots.
+ This function take an &odr; stream as parameter. This stream is used to
+ allocate memory for the data elements, which is released on a
+ subsequent call to <function>odr_reset()</function> on that stream.
+ </para>
+ <example id="tools.oid.bib1.3">
+ <title>Using odr_getoidbystr</title>
+ <para>
+ We can create a OID for the Bib-1 attribute set with:
+ <screen>
+ Odr_oid *bib1 = odr_getoidbystr(odr, "1.2.840.10003.3.1");
+ </screen>
+ </para>
+ </example>
+ <para>
+ The function
+ <screen>
+ char *oid_oid_to_dotstring(const Odr_oid *oid, char *oidbuf)
+ </screen>
+ does the reverse of <function>oid_oiddotstring_to_oid</function>. It
+ converts an OID to the string-based representation using dots.
+ The supplied char buffer <literal>oidbuf</literal> holds the resulting
+ string and must be at least <literal>OID_STR_MAX</literal> in size.
+ </para>
+ <para>
+ OIDs can be copied with <function>oid_oidcpy</function> which takes
+ two OID lists as arguments. Alternativly, an OID copy can be allocated
+ on a ODR stream with:
+ <screen>
+ Odr_oid *odr_oiddup(ODR odr, const Odr_oid *o);
+ </screen>
+ </para>
+ <para>
+ OIDs can be compared with <function>oid_oidcmp</function> which returns
+ zero if the two OIDs provided are identical; non-zero otherwise.
+ </para>
+ <sect2 id="tools.oid.database">
+ <title>OID database</title>
+ <para>
+ From YAZ version 3 and later, the oident system has been replaced
+ by an OID database. OID database is a misnomer .. the old odient
+ system was also a database.
+ </para>
+ <para>
+ The OID database is really just a map between named Object Identifiers
+ (string) and their OID raw equivalents. Most operations either
+ convert from string to OID or other way around.
+ </para>
+ <para>
+ Unfortunately, whenever we supply a string we must also specify the
+ <emphasis>OID class</emphasis>. The class is necessary because some
+ strings correspond to multiple OIDs. An example of such a string is
+ <literal>Bib-1</literal> which may either be an attribute-set
+ or a diagnostic-set.
+ </para>
+ <para>
+ Applications using the YAZ database should include
+ <filename>yaz/oid_db.h</filename>.
+ </para>
+ <para>
+ A YAZ database handle is of type <literal>yaz_oid_db_t</literal>.
+ Actually that's a pointer. You need not think deal with that.
+ YAZ has a built-in database which can be considered "constant" for
+ most purposes.
+ We can get hold that by using function <function>yaz_oid_std</function>.
+ </para>
+ <para>
+ All functions with prefix <function>yaz_string_to_oid</function>
+ converts from class + string to OID. We have variants of this
+ operation due to different memory allocation strategies.
+ </para>
+ <para>
+ All functions with prefix
+ <function>yaz_oid_to_string</function> converts from OID to string
+ + class.
+ </para>
+ <example id="tools.oid.bib1.4">
+ <title>Create OID with YAZ DB</title>
+ <para>
+ We can create an OID for the Bib-1 attribute set on the ODR stream
+ odr with:
+ <screen>
+ Odr_oid *bib1 =
+ yaz_string_to_oid_odr(yaz_oid_std(), CLASS_ATTSET, "Bib-1", odr);
+ </screen>
+ This is more complex than using <function>odr_getoidbystr</function>.
+ You would only use <function>yaz_string_to_oid_odr</function> when the
+ string (here Bib-1) is supplied by a user or configuration.
+ </para>
+ </example>
+ </sect2>
+ <sect2 id="tools.oid.std">
+ <title>Standard OIDs</title>
+ <para>
+ All the object identifers in the standard OID database as returned
+ by <function>yaz_oid_std</function> can referenced directly in a
+ program as a constant OID.
+ Each constant OID is prefixed with <literal>yaz_oid_</literal> -
+ followed by OID class (lowercase) - then by OID name (normalized and
+ lowercase).
+ </para>
+ <para>
+ See <xref linkend="list-oids"/> for list of all object identifiers
+ built into YAZ.
+ These are declared in <filename>yaz/oid_std.h</filename> but are
+ included by <filename>yaz/oid_db.h</filename> as well.
+ </para>
+ <example id="tools.oid.bib1.5">
+ <title>Use a built-in OID</title>
+ <para>
+ We can allocate our own OID filled with the constant OID for
+ Bib-1 with:
+ <screen>
+ Odr_oid *bib1 = odr_oiddup(o, yaz_oid_attset_bib1);
+ </screen>
+ </para>
+ </example>
+ </sect2>
+ </sect1>
+ <sect1 id="tools.nmem">
+ <title>Nibble Memory</title>
+ <para>
+ Sometimes when you need to allocate and construct a large,
+ interconnected complex of structures, it can be a bit of a pain to
+ release the associated memory again. For the structures describing the
+ Z39.50 PDUs and related structures, it is convenient to use the
+ memory-management system of the &odr; subsystem (see
+ <xref linkend="odr.use"/>). However, in some circumstances
+ where you might otherwise benefit from using a simple nibble memory
+ management system, it may be impractical to use
+ <function>odr_malloc()</function> and <function>odr_reset()</function>.
+ For this purpose, the memory manager which also supports the &odr;
+ streams is made available in the NMEM module. The external interface
+ to this module is given in the <filename>nmem.h</filename> file.
+ </para>
+ <para>
+ The following prototypes are given:
+ </para>
+ <screen>
+ NMEM nmem_create(void);
+ void nmem_destroy(NMEM n);
+ void *nmem_malloc(NMEM n, size_t size);
+ void nmem_reset(NMEM n);
+ size_t nmem_total(NMEM n);
+ void nmem_init(void);
+ void nmem_exit(void);
+ </screen>
+ <para>
+ The <function>nmem_create()</function> function returns a pointer to a
+ memory control handle, which can be released again by
+ <function>nmem_destroy()</function> when no longer needed.
+ The function <function>nmem_malloc()</function> allocates a block of
+ memory of the requested size. A call to <function>nmem_reset()</function>
+ or <function>nmem_destroy()</function> will release all memory allocated
+ on the handle since it was created (or since the last call to
+ <function>nmem_reset()</function>. The function
+ <function>nmem_total()</function> returns the number of bytes currently
+ allocated on the handle.
+ </para>
+ <para>
+ The nibble memory pool is shared amongst threads. POSIX
+ mutex'es and WIN32 Critical sections are introduced to keep the
+ module thread safe. Function <function>nmem_init()</function>
+ initializes the nibble memory library and it is called automatically
+ the first time the <literal>YAZ.DLL</literal> is loaded. &yaz; uses
+ function <function>DllMain</function> to achieve this. You should
+ <emphasis>not</emphasis> call <function>nmem_init</function> or
+ <function>nmem_exit</function> unless you're absolute sure what
+ you're doing. Note that in previous &yaz; versions you'd have to call
+ <function>nmem_init</function> yourself.
+ </para>
+ </sect1>
+ <sect1 id="tools.log">
+ <title>Log</title>
+ <para>
+ &yaz; has evolved a fairly complex log system which should be useful both
+ for debugging &yaz; itself, debugging applications that use &yaz;, and for
+ production use of those applications.
+ </para>
+ <para>
+ The log functions are declared in header <filename>yaz/log.h</filename>
+ and implemented in <filename>src/log.c</filename>.
+ Due to name clash with syslog and some math utilities the logging
+ interface has been modified as of YAZ 2.0.29. The obsolete interface
+ is still available if in header file <filename>yaz/log.h</filename>.
+ The key points of the interface are:
+ </para>
+ <screen>
+ void yaz_log(int level, const char *fmt, ...)
+ void yaz_log_init(int level, const char *prefix, const char *name);
+ void yaz_log_init_file(const char *fname);
+ void yaz_log_init_level(int level);
+ void yaz_log_init_prefix(const char *prefix);
+ void yaz_log_time_format(const char *fmt);
+ void yaz_log_init_max_size(int mx);
+
+ int yaz_log_mask_str(const char *str);
+ int yaz_log_module_level(const char *name);
+ </screen>
+ <para>
+ The reason for the whole log module is the <function>yaz_log</function>
+ function. It takes a bitmask indicating the log levels, a
+ <literal>printf</literal>-like format string, and a variable number of
+ arguments to log.
+ </para>
+ <para>
+ The <literal>log level</literal> is a bit mask, that says on which level(s)
+ the log entry should be made, and optionally set some behaviour of the
+ logging. In the most simple cases, it can be one of <literal>YLOG_FATAL,
+ YLOG_DEBUG, YLOG_WARN, YLOG_LOG</literal>. Those can be combined with bits
+ that modify the way the log entry is written:<literal>YLOG_ERRNO,
+ YLOG_NOTIME, YLOG_FLUSH</literal>.
+ Most of the rest of the bits are deprecated, and should not be used. Use
+ the dynamic log levels instead.
+ </para>
+ <para>
+ Applications that use &yaz;, should not use the LOG_LOG for ordinary
+ messages, but should make use of the dynamic loglevel system. This consists
+ of two parts, defining the loglevel and checking it.
+ </para>
+ <para>
+ To define the log levels, the (main) program should pass a string to
+ <function>yaz_log_mask_str</function> to define which log levels are to be
+ logged. This string should be a comma-separated list of log level names,
+ and can contain both hard-coded names and dynamic ones. The log level
+ calculation starts with <literal>YLOG_DEFAULT_LEVEL</literal> and adds a bit
+ for each word it meets, unless the word starts with a '-', in which case it
+ clears the bit. If the string <literal>'none'</literal> is found,
+ all bits are cleared. Typically this string comes from the command-line,
+ often identified by <literal>-v</literal>. The
+ <function>yaz_log_mask_str</function> returns a log level that should be
+ passed to <function>yaz_log_init_level</function> for it to take effect.
+ </para>
+ <para>
+ Each module should check what log bits it should be used, by calling
+ <function>yaz_log_module_level</function> with a suitable name for the
+ module. The name is cleared from a preceding path and an extension, if any,
+ so it is quite possible to use <literal>__FILE__</literal> for it. If the
+ name has been passed to <function>yaz_log_mask_str</function>, the routine
+ returns a non-zero bitmask, which should then be used in consequent calls
+ to yaz_log. (It can also be tested, so as to avoid unnecessary calls to
+ yaz_log, in time-critical places, or when the log entry would take time
+ to construct.)
+ </para>
+ <para>
+ Yaz uses the following dynamic log levels:
+ <literal>server, session, request, requestdetail</literal> for the server
+ functionality.
+ <literal>zoom</literal> for the zoom client api.
+ <literal>ztest</literal> for the simple test server.
+ <literal>malloc, nmem, odr, eventl</literal> for internal
+ debugging of yaz itself.
+ Of course, any program using yaz is welcome to define as many new
+ ones, as it needs.
+ </para>
+ <para>
+ By default the log is written to stderr, but this can be changed by a call
+ to <function>yaz_log_init_file</function> or
+ <function>yaz_log_init</function>. If the log is directed to a file, the
+ file size is checked at every write, and if it exceeds the limit given in
+ <function>yaz_log_init_max_size</function>, the log is rotated. The
+ rotation keeps one old version (with a <literal>.1</literal> appended to
+ the name). The size defaults to 1GB. Setting it to zero will disable the
+ rotation feature.
+ </para>
+ <screen>
+ A typical yaz-log looks like this
+ 13:23:14-23/11 yaz-ztest(1) [session] Starting session from tcp:127.0.0.1 (pid=30968)
+ 13:23:14-23/11 yaz-ztest(1) [request] Init from 'YAZ' (81) (ver 2.0.28) OK
+ 13:23:17-23/11 yaz-ztest(1) [request] Search Z: @attrset Bib-1 foo OK:7 hits
+ 13:23:22-23/11 yaz-ztest(1) [request] Present: [1] 2+2 OK 2 records returned
+ 13:24:13-23/11 yaz-ztest(1) [request] Close OK
+ </screen>
+ <para>
+ The log entries start with a time stamp. This can be omitted by setting the
+ <literal>YLOG_NOTIME</literal> bit in the loglevel. This way automatic tests
+ can be hoped to produce identical log files, that are easy to diff. The
+ format of the time stamp can be set with
+ <function>yaz_log_time_format</function>, which takes a format string just
+ like <function>strftime</function>.
+ </para>
+ <para>
+ Next in a log line comes the prefix, often the name of the program. For
+ yaz-based servers, it can also contain the session number. Then
+ comes one or more logbits in square brackets, depending on the logging
+ level set by <function>yaz_log_init_level</function> and the loglevel
+ passed to <function>yaz_log_init_level</function>. Finally comes the format
+ string and additional values passed to <function>yaz_log</function>
+ </para>
+ <para>
+ The log level <literal>YLOG_LOGLVL</literal>, enabled by the string
+ <literal>loglevel</literal>, will log all the log-level affecting
+ operations. This can come in handy if you need to know what other log
+ levels would be useful. Grep the logfile for <literal>[loglevel]</literal>.
+ </para>
+ <para>
+ The log system is almost independent of the rest of &yaz;, the only
+ important dependence is of <filename>nmem</filename>, and that only for
+ using the semaphore definition there.
+ </para>
+ <para>
+ The dynamic log levels and log rotation were introduced in &yaz; 2.0.28. At
+ the same time, the log bit names were changed from
+ <literal>LOG_something</literal> to <literal>YLOG_something</literal>,
+ to avoid collision with <filename>syslog.h</filename>.
+ </para>
+ </sect1>
+ <sect1 id="marc">
+ <title>MARC</title>
+ <para>
+ YAZ provides a fast utility for working with MARC records.
+ Early versions of the MARC utility only allowed decoding of ISO2709.
+ Today the utility may both encode - and decode to a varity of formats.
+ </para>
+ <synopsis><![CDATA[
+ #include <yaz/marcdisp.h>
+
+ /* create handler */
+ yaz_marc_t yaz_marc_create(void);
+ /* destroy */
+ void yaz_marc_destroy(yaz_marc_t mt);
+
+ /* set XML mode YAZ_MARC_LINE, YAZ_MARC_SIMPLEXML, ... */
+ void yaz_marc_xml(yaz_marc_t mt, int xmlmode);
+ #define YAZ_MARC_LINE 0
+ #define YAZ_MARC_SIMPLEXML 1
+ #define YAZ_MARC_OAIMARC 2
+ #define YAZ_MARC_MARCXML 3
+ #define YAZ_MARC_ISO2709 4
+ #define YAZ_MARC_XCHANGE 5
+ #define YAZ_MARC_CHECK 6
+ #define YAZ_MARC_TURBOMARC 7
+ #define YAZ_MARC_JSON 8
+
+ /* supply iconv handle for character set conversion .. */
+ void yaz_marc_iconv(yaz_marc_t mt, yaz_iconv_t cd);
+
+ /* set debug level, 0=none, 1=more, 2=even more, .. */
+ void yaz_marc_debug(yaz_marc_t mt, int level);
+
+ /* decode MARC in buf of size bsize. Returns >0 on success; <=0 on failure.
+ On success, result in *result with size *rsize. */
+ int yaz_marc_decode_buf(yaz_marc_t mt, const char *buf, int bsize,
+ const char **result, size_t *rsize);
+
+ /* decode MARC in buf of size bsize. Returns >0 on success; <=0 on failure.
+ On success, result in WRBUF */
+ int yaz_marc_decode_wrbuf(yaz_marc_t mt, const char *buf,
+ int bsize, WRBUF wrbuf);
+]]>
+ </synopsis>
+ <note>
+ <para>
+ The synopsis is just a basic subset of all functionality. Refer
+ to the actual header file <filename>marcdisp.h</filename> for
+ details.
+ </para>
+ </note>
+ <para>
+ A MARC conversion handle must be created by using
+ <function>yaz_marc_create</function> and destroyed
+ by calling <function>yaz_marc_destroy</function>.
+ </para>
+ <para>
+ All other function operate on a <literal>yaz_marc_t</literal> handle.
+ The output is specified by a call to <function>yaz_marc_xml</function>.
+ The <literal>xmlmode</literal> must be one of
+ <variablelist>
+ <varlistentry>
+ <term>YAZ_MARC_LINE</term>
+ <listitem>
+ <para>
+ A simple line-by-line format suitable for display but not
+ recommend for further (machine) processing.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_MARCXML</term>
+ <listitem>
+ <para>
+ <ulink url="&url.marcxml;">MARCXML</ulink>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_ISO2709</term>
+ <listitem>
+ <para>
+ ISO2709 (sometimes just referred to as "MARC").
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_XCHANGE</term>
+ <listitem>
+ <para>
+ <ulink url="&url.marcxchange;">MarcXchange</ulink>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_CHECK</term>
+ <listitem>
+ <para>
+ Pseudo format for validation only. Does not generate
+ any real output except diagnostics.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_TURBOMARC</term>
+ <listitem>
+ <para>
+ XML format with same semantics as MARCXML but more compact
+ and geared towards fast processing with XSLT. Refer to
+ <xref linkend="tools.turbomarc"/> for more information.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>YAZ_MARC_JSON</term>
+ <listitem>
+ <para>
+ <ulink url="&url.marc_in_json;">MARC-in_JSON</ulink> format.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ The actual conversion functions are
+ <function>yaz_marc_decode_buf</function> and
+ <function>yaz_marc_decode_wrbuf</function> which decodes and encodes
+ a MARC record. The former function operates on simple buffers, the
+ stores the resulting record in a WRBUF handle (WRBUF is a simple string
+ type).
+ </para>
+ <example id="example.marc.display">
+ <title>Display of MARC record</title>
+ <para>
+ The following program snippet illustrates how the MARC API may
+ be used to convert a MARC record to the line-by-line format:
+ <programlisting><![CDATA[
+ void print_marc(const char *marc_buf, int marc_buf_size)
+ {
+ char *result; /* for result buf */
+ size_t result_len; /* for size of result */
+ yaz_marc_t mt = yaz_marc_create();
+ yaz_marc_xml(mt, YAZ_MARC_LINE);
+ yaz_marc_decode_buf(mt, marc_buf, marc_buf_size,
+ &result, &result_len);
+ fwrite(result, result_len, 1, stdout);
+ yaz_marc_destroy(mt); /* note that result is now freed... */
+ }
+]]>
+ </programlisting>
+ </para>
+ </example>
+ <sect2 id="tools.turbomarc">
+ <title>TurboMARC</title>
+ <para>
+ TurboMARC is yet another XML encoding of a MARC record. The format
+ was designed for fast processing with XSLT.
+ </para>
+ <para>
+ Applications like
+ Pazpar2 uses XSLT to convert an XML encoded MARC record to an internal
+ representation. This conversion mostly check the tag of a MARC field
+ to determine the basic rules in the conversion. This check is
+ costly when that is tag is encoded as an attribute in MARCXML.
+ By having the tag value as the element instead, makes processing
+ many times faster (at least for Libxslt).
+ </para>
+ <para>
+ TurboMARC is encoded as follows:
+ <itemizedlist>
+ <listitem>
+ <para>
+ Record elements is part of namespace
+ "<literal>http://www.indexdata.com/turbomarc</literal>".
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ A record is enclosed in element <literal>r</literal>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ A collection of records is enclosed in element
+ <literal>collection</literal>.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The leader is encoded as element <literal>l</literal> with the
+ leader content as its (text) value.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ A control field is encoded as element <literal>c</literal> concatenated
+ with the tag value of the control field if the tag value
+ matches the regular expression <literal>[a-zA-Z0-9]*</literal>.
+ If the tag value do not match the regular expression
+ <literal>[a-zA-Z0-9]*</literal> the control field is encoded
+ as element <literal>c</literal> and attribute <literal>code</literal>
+ will hold the tag value.
+ This rule ensure that in the rare cases where a tag value might
+ result in a non-wellformed XML YAZ encode it as a coded attribute
+ (as in MARCXML).
+ </para>
+ <para>
+ The control field content is the the text value of this element.
+ Indicators are encoded as attribute names
+ <literal>i1</literal>, <literal>i2</literal>, etc.. and
+ corresponding values for each indicator.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ A data field is encoded as element <literal>d</literal> concatenated
+ with the tag value of the data field or using the attribute
+ <literal>code</literal> as described in the rules for control fields.
+ The children of the data field element is subfield elements.
+ Each subfield element is encoded as <literal>s</literal>
+ concatenated with the sub field code.
+ The text of the subfield element is the contents of the subfield.
+ Indicators are encoded as attributes for the data field element similar
+ to the encoding for control fields.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="tools.retrieval">
+ <title>Retrieval Facility</title>
+ <para>
+ YAZ version 2.1.20 or later includes a Retrieval facility tool
+ which allows a SRU/Z39.50 to describe itself and perform record
+ conversions. The idea is the following:
+ <itemizedlist>
+ <listitem>
+ <para>
+ An SRU/Z39.50 client sends a retrieval request which includes
+ a combination of the following parameters: syntax (format),
+ schema (or element set name).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The retrieval facility is invoked with parameters in a
+ server/proxy. The retrieval facility matches the parameters a set of
+ "supported" retrieval types.
+ If there is no match, the retrieval signals an error
+ (syntax and / or schema not supported).
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ For a successful match, the backend is invoked with the same
+ or altered retrieval parameters (syntax, schema). If
+ a record is received from the backend, it is converted to the
+ frontend name / syntax.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ The resulting record is sent back the client and tagged with
+ the frontend syntax / schema.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The Retrieval facility is driven by an XML configuration. The
+ configuration is neither Z39.50 ZeeRex or SRU ZeeRex. But it
+ should be easy to generate both of them from the XML configuration.
+ (unfortunately the two versions
+ of ZeeRex differ substantially in this regard).
+ </para>
+ <sect2 id="tools.retrieval.format">
+ <title>Retrieval XML format</title>
+ <para>
+ All elements should be covered by namespace
+ <literal>http://indexdata.com/yaz</literal> .
+ The root element node must be <literal>retrievalinfo</literal>.
+ </para>
+ <para>
+ The <literal>retrievalinfo</literal> must include one or
+ more <literal>retrieval</literal> elements. Each
+ <literal>retrieval</literal> defines specific combination of
+ syntax, name and identifier supported by this retrieval service.
+ </para>
+ <para>
+ The <literal>retrieval</literal> element may include any of the
+ following attributes:
+ <variablelist>
+ <varlistentry><term><literal>syntax</literal> (REQUIRED)</term>
+ <listitem>
+ <para>
+ Defines the record syntax. Possible values is any
+ of the names defined in YAZ' OID database or a raw
+ OID in (n.n ... n).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry><term><literal>name</literal> (OPTIONAL)</term>
+ <listitem>
+ <para>
+ Defines the name of the retrieval format. This can be
+ any string. For SRU, the value, is equivalent to schema (short-hand);
+ for Z39.50 it's equivalent to simple element set name.
+ For YAZ 3.0.24 and later this name may be specified as a glob
+ expression with operators
+ <literal>*</literal> and <literal>?</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry><term><literal>identifier</literal> (OPTIONAL)</term>
+ <listitem>
+ <para>
+ Defines the URI schema name of the retrieval format. This can be
+ any string. For SRU, the value, is equivalent to URI schema.
+ For Z39.50, there is no equivalent.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ <para>
+ The <literal>retrieval</literal> may include one
+ <literal>backend</literal> element. If a <literal>backend</literal>
+ element is given, it specifies how the records are retrieved by
+ some backend and how the records are converted from the backend to
+ the "frontend".
+ </para>
+ <para>
+ The attributes, <literal>name</literal> and <literal>syntax</literal>
+ may be specified for the <literal>backend</literal> element. These
+ semantics of these attributes is equivalent to those for the
+ <literal>retrieval</literal>. However, these values are passed to
+ the "backend".
+ </para>
+ <para>
+ The <literal>backend</literal> element may includes one or more
+ conversion instructions (as children elements). The supported
+ conversions are:
+ <variablelist>
+ <varlistentry><term><literal>marc</literal></term>
+ <listitem>
+ <para>
+ The <literal>marc</literal> element specifies a conversion
+ to - and from ISO2709 encoded MARC and
+ <ulink url="&url.marcxml;">&acro.marcxml;</ulink>/MarcXchange.
+ The following attributes may be specified:
+ <variablelist>
+ <varlistentry>
+ <term><literal>inputformat</literal> (REQUIRED)</term>
+ <listitem>
+ <para>
+ Format of input. Supported values are
+ <literal>marc</literal> (for ISO2709), <literal>xml</literal>
+ (MARCXML/MarcXchange) and <literal>json</literal>
+ (<ulink url="&url.marc_in_json;">MARC-in_JSON</ulink>).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>outputformat</literal> (REQUIRED)</term>
+ <listitem>
+ <para>
+ Format of output. Supported values are
+ <literal>line</literal> (MARC line format);
+ <literal>marcxml</literal> (for MARCXML),
+ <literal>marc</literal> (ISO2709),
+ <literal>marcxhcange</literal> (for MarcXchange),
+ or <literal>json</literal>
+ (<ulink url="&url.marc_in_json;">MARC-in_JSON </ulink>).
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>inputcharset</literal> (OPTIONAL)</term>
+ <listitem>
+ <para>
+ Encoding of input. For XML input formats, this need not
+ be given, but for ISO2709 based inputformats, this should
+ be set to the encoding used. For MARC21 records, a common
+ inputcharset value would be <literal>marc-8</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>outputcharset</literal> (OPTIONAL)</term>
+ <listitem>
+ <para>
+ Encoding of output. If outputformat is XML based, it is
+ strongly recommened to use <literal>utf-8</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>xslt</literal></term>
+ <listitem>
+ <para>
+ The <literal>xslt</literal> element specifies a conversion
+ via &acro.xslt;. The following attributes may be specified:
+ <variablelist>
+ <varlistentry><term><literal>stylesheet</literal> (REQUIRED)</term>
+ <listitem>
+ <para>
+ Stylesheet file.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>solrmarc</literal></term>
+ <listitem>
+ <para>
+ The <literal>solrmarc</literal> decodes solrmarc records.
+ It assumes that the input is pure solrmarc text (no escaping)
+ and will convert all sequences of the form #XX; to a single
+ character of the hexadecimal value as given by XX. The output,
+ presumably, is a valid ISO2709 buffer.
+ </para>
+ <para>
+ This conversion is available in YAZ 5.0.21 and later.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+ </sect2>
+ <sect2 id="tools.retrieval.examples">
+ <title>Retrieval Facility Examples</title>
+ <example id="tools.retrieval.marc21">
+ <title>MARC21 backend</title>
+ <para>
+ A typical way to use the retrieval facility is to enable XML
+ for servers that only supports ISO2709 encoded MARC21 records.
+ </para>
+ <programlisting><![CDATA[
+ <retrievalinfo>
+ <retrieval syntax="usmarc" name="F"/>
+ <retrieval syntax="usmarc" name="B"/>
+ <retrieval syntax="xml" name="marcxml"
+ identifier="info:srw/schema/1/marcxml-v1.1">
+ <backend syntax="usmarc" name="F">
+ <marc inputformat="marc" outputformat="marcxml"
+ inputcharset="marc-8"/>
+ </backend>
+ </retrieval>
+ <retrieval syntax="xml" name="dc">
+ <backend syntax="usmarc" name="F">
+ <marc inputformat="marc" outputformat="marcxml"
+ inputcharset="marc-8"/>
+ <xslt stylesheet="MARC21slim2DC.xsl"/>
+ </backend>
+ </retrieval>
+ </retrievalinfo>
+]]>
+ </programlisting>
+ <para>
+ This means that our frontend supports:
+ <itemizedlist>
+ <listitem>
+ <para>
+ MARC21 F(ull) records.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ MARC21 B(rief) records.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ MARCXML records.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Dublin core records.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </example>
+ <example id="tools.retrieval.marcxml">
+ <title>MARCXML backend</title>
+ <para>
+ SRW/SRU and Solr backends returns records in XML.
+ If they return MARCXML or MarcXchange, the retrieval module
+ can convert those into ISO2709 formats, most commonly USMARC
+ (AKA MARC21).
+ In this example, the backend returns MARCXML for schema="marcxml".
+ </para>
+ <programlisting><![CDATA[
+ <retrievalinfo>
+ <retrieval syntax="usmarc">
+ <backend syntax="xml" name="marcxml">
+ <marc inputformat="xml" outputformat="marc"
+ outputcharset="marc-8"/>
+ </backend>
+ </retrieval>
+ <retrieval syntax="xml" name="marcxml"
+ identifier="info:srw/schema/1/marcxml-v1.1"/>
+ <retrieval syntax="xml" name="dc">
+ <backend syntax="xml" name="marcxml">
+ <xslt stylesheet="MARC21slim2DC.xsl"/>
+ </backend>
+ </retrieval>
+ </retrievalinfo>
+]]>
+ </programlisting>
+ <para>
+ This means that our frontend supports:
+ <itemizedlist>
+ <listitem>
+ <para>
+ MARC21 records (any element set name) in MARC-8 encoding.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ MARCXML records for element-set=marcxml
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Dublin core records for element-set=dc.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </example>
+ </sect2>
+ <sect2 id="tools.retrieval.api">
+ <title>API</title>
+ <para>
+ It should be easy to use the retrieval systems from applications. Refer
+ to the headers
+ <filename>yaz/retrieval.h</filename> and
+ <filename>yaz/record_conv.h</filename>.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="sorting">
+ <title>Sorting</title>
+ <para>
+ This chapter describes sorting and how it is supported in YAZ.
+ Sorting applies to a result-set.
+ The
+ <ulink url="http://www.loc.gov/z3950/agency/markup/05.html#3.2.7">
+ Z39.50 sorting facility
+ </ulink>
+ takes one or more input result-sets
+ and one result-set as output. The most simple case is that
+ the input-set is the same as the output-set.
+ </para>
+ <para>
+ Z39.50 sorting has a separate APDU (service) that is, thus, performed
+ following a search (two phases).
+ </para>
+ <para>
+ In SRU/Solr, however, the model is different. Here, sorting is specified
+ during the the search operation. Note, however, that SRU might
+ perform sort as separate search, by referring to an existing result-set
+ in the query (result-set reference).
+ </para>
+ <sect2>
+ <title>Using the Z39.50 sort service</title>
+ <para>
+ yaz-client and the ZOOM API supports the Z39.50 sort facility. In any
+ case the sort sequence or sort critiera is using a string notation.
+ This notation is a one-line notation suitable for being manually
+ entered or generated and allows for easy logging (one liner).
+ For the ZOOM API, the sort is specified in the call to ZOOM_query_sortby
+ function. For yaz-client the sort is performed and specified using
+ the sort and sort+ commands. For description of the sort criteria notation
+ refer to the <link linkend="sortspec">sort command</link> in the
+ yaz-client manual.
+ </para>
+ <para>
+ The ZOOM API might choose one of several sort strategies for
+ sorting. Refer to <xref linkend="zoom-sort-strategy"/>.
+ </para>
+ </sect2>
+ <sect2>
+ <title>Type-7 sort</title>
+ <para>
+ Type-7 sort is an extension to the Bib-1 based RPN query where the
+ sort specification is embedded as an Attribute-Plus-Term.
+ </para>
+ <para>
+ The objectives for introducing Type-7 sorting is that it allows
+ a client to perform sorting even if it does not implement/support
+ Z39.50 sort. Virtually all Z39.50 client software supports
+ RPN queries. It also may improve performance because the sort
+ critieria is specified along with the search query.
+ </para>
+ <para>
+ The sort is triggered by the presence of type 7 and the value of type 7
+ specifies the
+ <ulink url="http://www.loc.gov/z3950/agency/asn1.html#SortKeySpec">
+ sortRelation
+ </ulink>
+ The value for type 7 is 1 for ascending and 2 for descending.
+ For the
+ <ulink url="http://www.loc.gov/z3950/agency/asn1.html#SortElement">
+ sortElement
+ </ulink>
+ only the generic part is handled. If generic sortKey is of type
+ sortField, then attribute type 1 is present and the value is
+ sortField (InternationalString). If generic sortKey is of type
+ sortAttributes, then the attributes in list is used . generic sortKey
+ of type elementSpec is not supported.
+ </para>
+ <para>
+ The term in the sorting Attribute-Plus-Term combo should hold
+ an integer. The value is 0 for primary sorting criteria, 1 for second
+ criteria, etc.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="facets">
+ <title>Facets</title>
+ <para>
+ YAZ supports facets for in Solr, SRU 2.0 and Z39.50 protocols.
+ </para>
+ <para>
+ Like Type-1/RPN, YAZ supports a string notation for specifying
+ facets. For the API this is performed by
+ <function>yaz_pqf_parse_facet_list</function>.
+ </para>
+ <para>
+ For ZOOM C the facets are given by option "facets"
+ For yaz-client it is used for the facets command.
+ </para>
+ <para>
+ The grammar of this specification is as follows:
+ <literallayout>
+ facet-spec ::= facet-list
+
+ facet-list ::= facet-list ',' attr-spec | attr-spec
+
+ attr-spec ::= attr-spec '@attr' string | '@attr' string
+
+ </literallayout>
+ The notation is inspired by PQF. The string following '@attr'
+ may not include blanks and is of the form
+ <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>,
+ where <replaceable>type</replaceable> is an integer and
+ <replaceable>value</replaceable> is a string or an integer.
+ </para>
+ <para>
+ The Facets specification is not Bib-1. The following types apply:
+ </para>
+ <table id="facet.attributes">
+ <title>Facet attributes</title>
+ <tgroup cols="2">
+ <colspec colwidth="2*" colname="type"></colspec>
+ <colspec colwidth="9*" colname="description"></colspec>
+ <thead>
+ <row>
+ <entry>Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>1</entry>
+ <entry>
+ Field-name. This is often a string, eg "Author", "Year", etc.
+ </entry>
+ </row>
+ <row>
+ <entry>2</entry>
+ <entry>
+ Sort order. Value should be an integer.
+ Value 0: count descending (frequency). Value 1: alpha ascending.
+ </entry>
+ </row>
+ <row>
+ <entry>3</entry>
+ <entry>
+ Number of terms requested.
+ </entry>
+ </row>
+ <row>
+ <entry>4</entry>
+ <entry>
+ Start offset.
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect1>
+ </chapter>
+ <chapter id="odr">
+ <title>The ODR Module</title>
+ <sect1 id="odr.introduction">
+ <title>Introduction</title>
+ <para>
+ &odr; is the BER-encoding/decoding subsystem of &yaz;. Care as been taken
+ to isolate &odr; from the rest of the package - specifically from the
+ transport interface. &odr; may be used in any context where basic
+ ASN.1/BER representations are used.
+ </para>
+ <para>
+ If you are only interested in writing a Z39.50 implementation based on
+ the PDUs that are already provided with &yaz;, you only need to concern
+ yourself with the section on managing ODR streams
+ (<xref linkend="odr.use"/>). Only if you need to
+ implement ASN.1 beyond that which has been provided, should you
+ worry about the second half of the documentation
+ (<xref linkend="odr.programming"/>).
+ If you use one of the higher-level interfaces, you can skip this
+ section entirely.
+ </para>
+ <para>
+ This is important, so we'll repeat it for emphasis: <emphasis>You do
+ not need to read <xref linkend="odr.programming"/>
+ to implement Z39.50 with &yaz;.</emphasis>
+ </para>
+ <para>
+ If you need a part of the protocol that isn't already in &yaz;, you
+ should contact the authors before going to work on it yourself: We
+ might already be working on it. Conversely, if you implement a useful
+ part of the protocol before us, we'd be happy to include it in a
+ future release.
+ </para>
+ </sect1>
+ <sect1 id="odr.use">
+ <title>Using ODR</title>
+ <sect2 id="odr.streams">
+ <title>ODR Streams</title>
+ <para>
+ Conceptually, the ODR stream is the source of encoded data in the
+ decoding mode; when encoding, it is the receptacle for the encoded
+ data. Before you can use an ODR stream it must be allocated. This is
+ done with the function
+ </para>
+ <synopsis>
+ ODR odr_createmem(int direction);
+ </synopsis>
+ <para>
+ The <function>odr_createmem()</function> function takes as argument one
+ of three manifest constants: <literal>ODR_ENCODE</literal>,
+ <literal>ODR_DECODE</literal>, or <literal>ODR_PRINT</literal>.
+ An &odr; stream can be in only one mode - it is not possible to change
+ its mode once it's selected. Typically, your program will allocate
+ at least two ODR streams - one for decoding, and one for encoding.
+ </para>
+ <para>
+ When you're done with the stream, you can use
+ </para>
+ <synopsis>
+ void odr_destroy(ODR o);
+ </synopsis>
+ <para>
+ to release the resources allocated for the stream.
+ </para>
+ </sect2>
+ <sect2 id="odr.memory.management">
+ <title id="memory">Memory Management</title>
+ <para>
+ Two forms of memory management take place in the &odr; system. The first
+ one, which has to do with allocating little bits of memory (sometimes
+ quite large bits of memory, actually) when a protocol package is
+ decoded, and turned into a complex of interlinked structures. This
+ section deals with this system, and how you can use it for your own
+ purposes. The next section deals with the memory management which is
+ required when encoding data - to make sure that a large enough buffer is
+ available to hold the fully encoded PDU.
+ </para>
+ <para>
+ The &odr; module has its own memory management system, which is
+ used whenever memory is required. Specifically, it is used to allocate
+ space for data when decoding incoming PDUs. You can use the memory
+ system for your own purposes, by using the function
+ </para>
+ <synopsis>
+ void *odr_malloc(ODR o, size_t size);
+ </synopsis>
+ <para>
+ You can't use the normal <function>free(2)</function> routine to free
+ memory allocated by this function, and &odr; doesn't provide a parallel
+ function. Instead, you can call
+ </para>
+ <synopsis>
+ void odr_reset(ODR o);
+ </synopsis>
+ <para>
+ when you are done with the
+ memory: Everything allocated since the last call to
+ <function>odr_reset()</function> is released.
+ The <function>odr_reset()</function> call is also required to clear
+ up an error condition on a stream.
+ </para>
+ <para>
+ The function
+ </para>
+ <synopsis>
+ size_t odr_total(ODR o);
+ </synopsis>
+ <para>
+ returns the number of bytes allocated on the stream since the last call to
+ <function>odr_reset()</function>.
+ </para>
+ <para>
+ The memory subsystem of &odr; is fairly efficient at allocating and
+ releasing little bits of memory. Rather than managing the individual,
+ small bits of space, the system maintains a free-list of larger chunks
+ of memory, which are handed out in small bits. This scheme is
+ generally known as a <emphasis>nibble memory</emphasis> system.
+ It is very useful for maintaining short-lived constructions such
+ as protocol PDUs.
+ </para>
+ <para>
+ If you want to retain a bit of memory beyond the next call to
+ <function>odr_reset()</function>, you can use the function
+ </para>
+ <synopsis>
+ ODR_MEM odr_extract_mem(ODR o);
+ </synopsis>
+ <para>
+ This function will give you control of the memory recently allocated
+ on the ODR stream. The memory will live (past calls to
+ <function>odr_reset()</function>), until you call the function
+ </para>
+ <synopsis>
+ void odr_release_mem(ODR_MEM p);
+ </synopsis>
+ <para>
+ The opaque <literal>ODR_MEM</literal> handle has no other purpose than
+ referencing the memory block for you until you want to release it.
+ </para>
+ <para>
+ You can use <function>odr_extract_mem()</function> repeatedly between
+ allocating data, to retain individual control of separate chunks of data.
+ </para>
+ </sect2>
+ <sect2 id="odr.encoding.and.decoding">
+ <title>Encoding and Decoding Data</title>
+ <para>
+ When encoding data, the ODR stream will write the encoded octet string
+ in an internal buffer. To retrieve the data, use the function
+ </para>
+ <synopsis>
+ char *odr_getbuf(ODR o, int *len, int *size);
+ </synopsis>
+ <para>
+ The integer pointed to by len is set to the length of the encoded
+ data, and a pointer to that data is returned. <literal>*size</literal>
+ is set to the size of the buffer (unless <literal>size</literal> is null,
+ signaling that you are not interested in the size). The next call to
+ a primitive function using the same &odr; stream will overwrite the
+ data, unless a different buffer has been supplied using the call
+ </para>
+ <synopsis>
+ void odr_setbuf(ODR o, char *buf, int len, int can_grow);
+ </synopsis>
+ <para>
+ which sets the encoding (or decoding) buffer used by
+ <literal>o</literal> to <literal>buf</literal>, using the length
+ <literal>len</literal>.
+ Before a call to an encoding function, you can use
+ <function>odr_setbuf()</function> to provide the stream with an encoding
+ buffer of sufficient size (length). The <literal>can_grow</literal>
+ parameter tells the encoding &odr; stream whether it is allowed to use
+ <function>realloc(2)</function> to increase the size of the buffer when
+ necessary. The default condition of a new encoding stream is equivalent
+ to the results of calling
+ </para>
+ <synopsis>
+ odr_setbuf(stream, 0, 0, 1);
+ </synopsis>
+ <para>
+ In this case, the stream will allocate and reallocate memory as
+ necessary. The stream reallocates memory by repeatedly doubling the
+ size of the buffer - the result is that the buffer will typically
+ reach its maximum, working size with only a small number of reallocation
+ operations. The memory is freed by the stream when the latter is destroyed,
+ unless it was assigned by the user with the <literal>can_grow</literal>
+ parameter set to zero (in this case, you are expected to retain
+ control of the memory yourself).
+ </para>
+ <para>
+ To assume full control of an encoded buffer, you must first call
+ <function>odr_getbuf()</function> to fetch the buffer and its length.
+ Next, you should call <function>odr_setbuf()</function> to provide a
+ different buffer (or a null pointer) to the stream. In the simplest
+ case, you will reuse the same buffer over and over again, and you
+ will just need to call <function>odr_getbuf()</function> after each
+ encoding operation to get the length and address of the buffer.
+ Note that the stream may reallocate the buffer during an encoding
+ operation, so it is necessary to retrieve the correct address after
+ each encoding operation.
+ </para>
+ <para>
+ It is important to realize that the ODR stream will not release this
+ memory when you call <function>odr_reset()</function>: It will
+ merely update its internal pointers to prepare for the encoding of a
+ new data value.
+ When the stream is released by the <function>odr_destroy()</function>
+ function, the memory given to it by <function>odr_setbuf</function> will
+ be released <emphasis>only</emphasis> if the <literal>can_grow</literal>
+ parameter to <function>odr_setbuf()</function> was nonzero. The
+ <literal>can_grow</literal> parameter, in other words, is a way of
+ signaling who is to own the buffer, you or the ODR stream. If you never call
+ <function>odr_setbuf()</function> on your encoding stream, which is
+ typically the case, the buffer allocated by the stream will belong to
+ the stream by default.
+ </para>
+ <para>
+ When you wish to decode data, you should first call
+ <function>odr_setbuf()</function>, to tell the decoding stream
+ where to find the encoded data, and how long the buffer is
+ (the <literal>can_grow</literal> parameter is ignored by a decoding
+ stream). After this, you can call the function corresponding to the
+ data you wish to decode (eg, <function>odr_integer()</function> odr
+ <function>z_APDU()</function>).
+ </para>
+ <example id="example.odr.encoding.and.decoding.functions">
+ <title>Encoding and decoding functions</title>
+ <synopsis>
+ int odr_integer(ODR o, Odr_int **p, int optional, const char *name);
+
+ int z_APDU(ODR o, Z_APDU **p, int optional, const char *name);
+ </synopsis>
+ </example>
+ <para>
+ If the data is absent (or doesn't match the tag corresponding to
+ the type), the return value will be either 0 or 1 depending on the
+ <literal>optional</literal> flag. If <literal>optional</literal>
+ is 0 and the data is absent, an error flag will be raised in the
+ stream, and you'll need to call <function>odr_reset()</function> before
+ you can use the stream again. If <literal>optional</literal> is
+ nonzero, the pointer <emphasis>pointed</emphasis> to/ by
+ <literal>p</literal> will be set to the null value, and the function
+ will return 1.
+ The <literal>name</literal> argument is used to pretty-print the
+ tag in question. It may be set to <literal>NULL</literal> if
+ pretty-printing is not desired.
+ </para>
+ <para>
+ If the data value is found where it's expected, the pointer
+ <emphasis>pointed to</emphasis> by the <literal>p</literal> argument
+ will be set to point to the decoded type.
+ The space for the type will be allocated and owned by the &odr;
+ stream, and it will live until you call
+ <function>odr_reset()</function> on the stream. You cannot use
+ <function>free(2)</function> to release the memory.
+ You can decode several data elements (by repeated calls to
+ <function>odr_setbuf()</function> and your decoding function), and
+ new memory will be allocated each time. When you do call
+ <function>odr_reset()</function>, everything decoded since the
+ last call to <function>odr_reset()</function> will be released.
+ </para>
+ <example id="example.odr.encoding.of.integer">
+ <title>Encoding and decoding of an integer</title>
+ <para>
+ The use of the double indirection can be a little confusing at first
+ (its purpose will become clear later on, hopefully),
+ so an example is in order. We'll encode an integer value, and
+ immediately decode it again using a different stream. A useless, but
+ informative operation.
+ </para>
+ <programlisting><![CDATA[
+void do_nothing_useful(Odr_int value)
+{
+ ODR encode, decode;
+ Odr_int *valp, *resvalp;
+ char *bufferp;
+ int len;
+
+ /* allocate streams */
+ if (!(encode = odr_createmem(ODR_ENCODE)))
+ return;
+ if (!(decode = odr_createmem(ODR_DECODE)))
+ return;
+
+ valp = &value;
+ if (odr_integer(encode, &valp, 0, 0) == 0)
+ {
+ printf("encoding went bad\n");
+ return;
+ }
+ bufferp = odr_getbuf(encode, &len, 0);
+ printf("length of encoded data is %d\n", len);
+
+ /* now let's decode the thing again */
+ odr_setbuf(decode, bufferp, len, 0);
+ if (odr_integer(decode, &resvalp, 0, 0) == 0)
+ {
+ printf("decoding went bad\n");
+ return;
+ }
+ /* ODR_INT_PRINTF format for printf (such as %d) */
+ printf("the value is " ODR_INT_PRINTF "\n", *resvalp);
+
+ /* clean up */
+ odr_destroy(encode);
+ odr_destroy(decode);
+}
+]]>
+ </programlisting>
+ <para>
+ This looks like a lot of work, offhand. In practice, the &odr; streams
+ will typically be allocated once, in the beginning of your program
+ (or at the beginning of a new network session), and the encoding
+ and decoding will only take place in a few, isolated places in your
+ program, so the overhead is quite manageable.
+ </para>
+ </example>
+ </sect2>
+ <sect2 id="odr.printing">
+ <title>Printing</title>
+ <para>
+ When an ODR stream is created of type <literal>ODR_PRINT</literal>
+ the ODR module will print the contents of a PDU in a readable format.
+ By default output is written to the <literal>stderr</literal> stream.
+ This behavior can be changed, however, by calling the function
+ <synopsis>
+ odr_setprint(ODR o, FILE *file);
+ </synopsis>
+ before encoders or decoders are being invoked.
+ It is also possible to direct the output to a buffer (of indeed
+ another file), by using the more generic mechanism:
+ <synopsis>
+ void odr_set_stream(ODR o, void *handle,
+ void (*stream_write)(ODR o, void *handle, int type,
+ const char *buf, int len),
+ void (*stream_close)(void *handle));
+ </synopsis>
+ Here the user provides an opaque handle and two handlers,
+ <replaceable>stream_write</replaceable> for writing,
+ and <replaceable>stream_close</replaceable> which is supposed
+ to close/free resources associated with handle.
+ The <replaceable>stream_close</replaceable> handler is optional and
+ if NULL for the function is provided, it will not be invoked.
+ The <replaceable>stream_write</replaceable> takes the ODR handle
+ as parameter, the user defined handle, a type
+ <literal>ODR_OCTETSTRING</literal>, <literal>ODR_VISIBLESTRING</literal>
+ which indicates the type of contents is being written.
+ </para>
+ <para>
+ Another utility useful for diagnostics (error handling) or as
+ part of the printing facilities is:
+ <synopsis>
+ const char **odr_get_element_path(ODR o);
+ </synopsis>
+ which returns a list of current elements that ODR deals with at the
+ moment. For the returned array, say <literal>ar</literal>,
+ <literal>ar[0]</literal> is the top level element,
+ <literal>ar[n]</literal> is the last. The last element has the
+ property that <literal>ar[n+1] == NULL</literal>.
+ </para>
+ <example id="example.odr.element.path.record">
+ <title>Element Path for record</title>
+ <para>
+ For a database record part of a PresentResponse the
+ array returned by <function>odr_get_element</function>
+ is <literal>presentResponse</literal>, <literal>databaseOrSurDiagnostics</literal>, <literal>?</literal>, <literal>record</literal>, <literal>?</literal>, <literal>databaseRecord</literal> . The question mark appears due to
+ unnamed constructions.
+ </para>
+ </example>
+ </sect2>
+ <sect2 id="odr.diagnostics">
+ <title>Diagnostics</title>
+ <para>
+ The encoding/decoding functions all return 0 when an error occurs.
+ Until you call <function>odr_reset()</function>, you cannot use the
+ stream again, and any function called will immediately return 0.
+ </para>
+ <para>
+ To provide information to the programmer or administrator, the function
+ </para>
+ <synopsis>
+ void odr_perror(ODR o, char *message);
+ </synopsis>
+ <para>
+ is provided, which prints the <literal>message</literal> argument to
+ <literal>stderr</literal> along with an error message from the stream.
+ </para>
+ <para>
+ You can also use the function
+ </para>
+ <synopsis>
+ int odr_geterror(ODR o);
+ </synopsis>
+ <para>
+ to get the current error number from the screen. The number will be
+ one of these constants:
+ </para>
+ <table frame="top" id="odr.error.codes">
+ <title>ODR Error codes</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>code</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>OMEMORY</entry><entry>Memory allocation failed.</entry>
+ </row>
+ <row>
+ <entry>OSYSERR</entry><entry>A system- or library call has failed.
+ The standard diagnostic variable <literal>errno</literal> should be
+ examined to determine the actual error.</entry>
+ </row>
+ <row>
+ <entry>OSPACE</entry><entry>No more space for encoding.
+ This will only occur when the user has explicitly provided a
+ buffer for an encoding stream without allowing the system to
+ allocate more space.</entry>
+ </row>
+ <row>
+ <entry>OREQUIRED</entry><entry>This is a common protocol error; A
+ required data element was missing during encoding or decoding.</entry>
+ </row>
+ <row>
+ <entry>OUNEXPECTED</entry><entry>An unexpected data element was
+ found during decoding.</entry>
+ </row>
+ <row>
+ <entry>OOTHER</entry><entry>Other error. This is typically an
+ indication of misuse of the &odr; system by the programmer, and also
+ that the diagnostic system isn't as good as it should be, yet.</entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ The character string array
+ </para>
+ <synopsis>
+ char *odr_errlist[]
+ </synopsis>
+ <para>
+ can be indexed by the error code to obtain a human-readable
+ representation of the problem.
+ </para>
+ </sect2>
+ <sect2 id="odr.summary.and.synopsis">
+ <title>Summary and Synopsis</title>
+ <synopsis>
+ #include <yaz/odr.h>
+
+ ODR odr_createmem(int direction);
+
+ void odr_destroy(ODR o);
+
+ void odr_reset(ODR o);
+
+ char *odr_getbuf(ODR o, int *len, int *size);
+
+ void odr_setbuf(ODR o, char *buf, int len, int can_grow);
+
+ void *odr_malloc(ODR o, int size);
+
+ NMEM odr_extract_mem(ODR o);
+
+ int odr_geterror(ODR o);
+
+ void odr_perror(ODR o, const char *message);
+
+ extern char *odr_errlist[];
+ </synopsis>
+ </sect2>
+ </sect1>
+ <sect1 id="odr.programming">
+ <title>Programming with ODR</title>
+ <para>
+ The API of &odr; is designed to reflect the structure of ASN.1, rather
+ than BER itself. Future releases may be able to represent data in
+ other external forms.
+ </para>
+ <tip>
+ <para>
+ There is an ASN.1 tutorial available at
+ <ulink url="&url.asn.1.tutorial;">this site</ulink>.
+ This site also has standards for ASN.1 (X.680) and BER (X.690)
+ <ulink url="&url.asn.1.standards;">online</ulink>.
+ </para>
+ </tip>
+ <para>
+ The ODR interface is based loosely on that of the Sun Microsystems
+ XDR routines.
+ Specifically, each function which corresponds to an ASN.1 primitive
+ type has a dual function. Depending on the settings of the ODR
+ stream which is supplied as a parameter, the function may be used
+ either to encode or decode data. The functions that can be built
+ using these primitive functions, to represent more complex data types,
+ share this quality. The result is that you only have to enter the
+ definition for a type once - and you have the functionality of encoding,
+ decoding (and pretty-printing) all in one unit.
+ The resulting C source code is quite compact, and is a pretty
+ straightforward representation of the source ASN.1 specification.
+ </para>
+ <para>
+ In many cases, the model of the XDR functions works quite well in this
+ role.
+ In others, it is less elegant. Most of the hassle comes from the optional
+ SEQUENCE members which don't exist in XDR.
+ </para>
+ <sect2 id="odr.primitive.asn1.types">
+ <title>The Primitive ASN.1 Types</title>
+ <para>
+ ASN.1 defines a number of primitive types (many of which correspond
+ roughly to primitive types in structured programming languages, such as C).
+ </para>
+ <sect3 id="odr.integer">
+ <title>INTEGER</title>
+ <para>
+ The &odr; function for encoding or decoding (or printing) the ASN.1
+ INTEGER type looks like this:
+ </para>
+ <synopsis>
+ int odr_integer(ODR o, Odr_int **p, int optional, const char *name);
+ </synopsis>
+ <para>
+ The <literal>Odr_int</literal> is just a simple integer.
+ </para>
+ <para>
+ This form is typical of the primitive &odr; functions. They are named
+ after the type of data that they encode or decode. They take an &odr;
+ stream, an indirect reference to the type in question, and an
+ <literal>optional</literal> flag (corresponding to the OPTIONAL keyword
+ of ASN.1) as parameters. They all return an integer value of either one
+ or zero.
+ When you use the primitive functions to construct encoders for complex
+ types of your own, you should follow this model as well. This
+ ensures that your new types can be reused as elements in yet more
+ complex types.
+ </para>
+ <para>
+ The <literal>o</literal> parameter should obviously refer to a properly
+ initialized &odr; stream of the right type (encoding/decoding/printing)
+ for the operation that you wish to perform.
+ </para>
+ <para>
+ When encoding or printing, the function first looks at
+ <literal>* p</literal>. If <literal>* p</literal> (the pointer pointed
+ to by <literal>p</literal>) is a null pointer, this is taken to mean that
+ the data element is absent. If the <literal>optional</literal> parameter
+ is nonzero, the function will return one (signifying success) without
+ any further processing. If the <literal>optional</literal> is zero, an
+ internal error flag is set in the &odr; stream, and the function will
+ return 0. No further operations can be carried out on the stream without
+ a call to the function <function>odr_reset()</function>.
+ </para>
+ <para>
+ If <literal>*p</literal> is not a null pointer, it is expected to
+ point to an instance of the data type. The data will be subjected to
+ the encoding rules, and the result will be placed in the buffer held
+ by the &odr; stream.
+ </para>
+ <para>
+ The other ASN.1 primitives have similar functions that operate in
+ similar manners:
+ </para>
+ </sect3>
+ <sect3 id="odr.boolean">
+ <title>BOOLEAN</title>
+ <synopsis>
+int odr_bool(ODR o, Odr_bool **p, int optional, const char *name);
+ </synopsis>
+ </sect3>
+ <sect3 id="odr.real">
+ <title>REAL</title>
+ <para>
+ Not defined.
+ </para>
+ </sect3>
+ <sect3 id="odr.null">
+ <title>NULL</title>
+ <synopsis>
+int odr_null(ODR o, Odr_null **p, int optional, const char *name);
+ </synopsis>
+ <para>
+ In this case, the value of **p is not important. If <literal>*p</literal>
+ is different from the null pointer, the null value is present, otherwise
+ it's absent.
+ </para>
+ </sect3>
+ <sect3 id="odr.octet.string">
+ <title>OCTET STRING</title>
+ <synopsis>
+typedef struct odr_oct
+{
+ unsigned char *buf;
+ int len;
+} Odr_oct;
+
+int odr_octetstring(ODR o, Odr_oct **p, int optional,
+ const char *name);
+ </synopsis>
+ <para>
+ The <literal>buf</literal> field should point to the character array
+ that holds the octetstring. The <literal>len</literal> field holds the
+ actual length.
+ The character array need not be null terminated.
+ </para>
+ <para>
+ To make things a little easier, an alternative is given for string
+ types that are not expected to contain embedded NULL characters (eg.
+ VisibleString):
+ </para>
+ <synopsis>
+ int odr_cstring(ODR o, char **p, int optional, const char *name);
+ </synopsis>
+ <para>
+ Which encoded or decodes between OCTETSTRING representations and
+ null-terminates C strings.
+ </para>
+ <para>
+ Functions are provided for the derived string types, eg:
+ </para>
+ <synopsis>
+int odr_visiblestring(ODR o, char **p, int optional,
+ const char *name);
+ </synopsis>
+ </sect3>
+ <sect3 id="odr.bit.string">
+ <title>BIT STRING</title>
+ <synopsis>
+int odr_bitstring(ODR o, Odr_bitmask **p, int optional,
+ const char *name);
+ </synopsis>
+ <para>
+ The opaque type <literal>Odr_bitmask</literal> is only suitable for
+ holding relatively brief bit strings, eg. for options fields, etc.
+ The constant <literal>ODR_BITMASK_SIZE</literal> multiplied by 8
+ gives the maximum possible number of bits.
+ </para>
+ <para>
+ A set of macros are provided for manipulating the
+ <literal>Odr_bitmask</literal> type:
+ </para>
+ <synopsis>
+void ODR_MASK_ZERO(Odr_bitmask *b);
+
+void ODR_MASK_SET(Odr_bitmask *b, int bitno);
+
+void ODR_MASK_CLEAR(Odr_bitmask *b, int bitno);
+
+int ODR_MASK_GET(Odr_bitmask *b, int bitno);
+ </synopsis>
+ <para>
+ The functions are modeled after the manipulation functions that
+ accompany the <literal>fd_set</literal> type used by the
+ <function>select(2)</function> call.
+ <literal>ODR_MASK_ZERO</literal> should always be called first on a
+ new bitmask, to initialize the bits to zero.
+ </para>
+ </sect3>
+ <sect3 id="odr.object.identifier">
+ <title>OBJECT IDENTIFIER</title>
+ <synopsis>
+int odr_oid(ODR o, Odr_oid **p, int optional, const char *name);
+ </synopsis>
+ <para>
+ The C OID representation is simply an array of integers, terminated by
+ the value -1 (the <literal>Odr_oid</literal> type is synonymous with
+ the <literal>short</literal> type).
+ We suggest that you use the OID database module (see
+ <xref linkend="tools.oid.database"/>) to handle object identifiers
+ in your application.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2 id="odr.tagging.primitive.types">
+ <title>Tagging Primitive Types</title>
+ <para>
+ The simplest way of tagging a type is to use the
+ <function>odr_implicit_tag()</function> or
+ <function>odr_explicit_tag()</function> macros:
+ </para>
+ <synopsis>
+int odr_implicit_tag(ODR o, Odr_fun fun, int class, int tag,
+ int optional, const char *name);
+
+int odr_explicit_tag(ODR o, Odr_fun fun, int class, int tag,
+ int optional, const char *name);
+ </synopsis>
+ <para>
+ To create a type derived from the integer type by implicit tagging, you
+ might write:
+ </para>
+ <screen>
+ MyInt ::= [210] IMPLICIT INTEGER
+ </screen>
+ <para>
+ In the &odr; system, this would be written like:
+ </para>
+ <screen>
+int myInt(ODR o, Odr_int **p, int optional, const char *name)
+{
+ return odr_implicit_tag(o, odr_integer, p,
+ ODR_CONTEXT, 210, optional, name);
+}
+ </screen>
+ <para>
+ The function <function>myInt()</function> can then be used like any of
+ the primitive functions provided by &odr;. Note that the behavior of
+ <function>odr_explicit_tag()</function>
+ and <function>odr_implicit_tag()</function> macros
+ act exactly the same as the functions they are applied to - they
+ respond to error conditions, etc, in the same manner - they
+ simply have three extra parameters. The class parameter may
+ take one of the values: <literal>ODR_CONTEXT</literal>,
+ <literal>ODR_PRIVATE</literal>, <literal>ODR_UNIVERSAL</literal>, or
+ <literal>/ODR_APPLICATION</literal>.
+ </para>
+ </sect2>
+ <sect2 id="odr.constructed.types">
+ <title>Constructed Types</title>
+ <para>
+ Constructed types are created by combining primitive types. The
+ &odr; system only implements the SEQUENCE and SEQUENCE OF constructions
+ (although adding the rest of the container types should be simple
+ enough, if the need arises).
+ </para>
+ <para>
+ For implementing SEQUENCEs, the functions
+ </para>
+ <synopsis>
+int odr_sequence_begin(ODR o, void *p, int size, const char *name);
+int odr_sequence_end(ODR o);
+ </synopsis>
+ <para>
+ are provided.
+ </para>
+ <para>
+ The <function>odr_sequence_begin()</function> function should be
+ called in the beginning of a function that implements a SEQUENCE type.
+ Its parameters are the &odr; stream, a pointer (to a pointer to the type
+ you're implementing), and the <literal>size</literal> of the type
+ (typically a C structure). On encoding, it returns 1 if
+ <literal>* p</literal> is a null pointer. The <literal>size</literal>
+ parameter is ignored. On decoding, it returns 1 if the type is found in
+ the data stream. <literal>size</literal> bytes of memory are allocated,
+ and <literal>*p</literal> is set to point to this space.
+ <function>odr_sequence_end()</function> is called at the end of the
+ complex function. Assume that a type is defined like this:
+ </para>
+ <screen>
+MySequence ::= SEQUENCE {
+ intval INTEGER,
+ boolval BOOLEAN OPTIONAL
+}
+ </screen>
+ <para>
+ The corresponding &odr; encoder/decoder function and the associated data
+ structures could be written like this:
+ </para>
+ <screen>
+typedef struct MySequence
+{
+ Odr_int *intval;
+ Odr_bool *boolval;
+} MySequence;
+
+int mySequence(ODR o, MySequence **p, int optional, const char *name)
+{
+ if (odr_sequence_begin(o, p, sizeof(**p), name) == 0)
+ return optional && odr_ok(o);
+ return
+ odr_integer(o, &(*p)->intval, 0, "intval") &&
+ odr_bool(o, &(*p)->boolval, 1, "boolval") &&
+ odr_sequence_end(o);
+}
+ </screen>
+ <para>
+ Note the 1 in the call to <function>odr_bool()</function>, to mark
+ that the sequence member is optional.
+ If either of the member types had been tagged, the macros
+ <function>odr_implicit_tag()</function> or
+ <function>odr_explicit_tag()</function>
+ could have been used.
+ The new function can be used exactly like the standard functions provided
+ with &odr;. It will encode, decode or pretty-print a data value of the
+ <literal>MySequence</literal> type. We like to name types with an
+ initial capital, as done in ASN.1 definitions, and to name the
+ corresponding function with the first character of the name in lower case.
+ You could, of course, name your structures, types, and functions any way
+ you please - as long as you're consistent, and your code is easily readable.
+ <literal>odr_ok</literal> is just that - a predicate that returns the
+ state of the stream. It is used to ensure that the behavior of the new
+ type is compatible with the interface of the primitive types.
+ </para>
+ </sect2>
+ <sect2 id="odr.tagging.constructed.types">
+ <title>Tagging Constructed Types</title>
+ <note>
+ <para>
+ See <xref linkend="odr.tagging.primitive.types"/> for information
+ on how to tag the primitive types, as well as types that are
+ already defined.
+ </para>
+ </note>
+ <sect3 id="odr.implicit.tagging">
+ <title>Implicit Tagging</title>
+ <para>
+ Assume the type above had been defined as
+ </para>
+ <screen>
+MySequence ::= [10] IMPLICIT SEQUENCE {
+ intval INTEGER,
+ boolval BOOLEAN OPTIONAL
+}
+ </screen>
+ <para>
+ You would implement this in &odr; by calling the function
+ </para>
+ <synopsis>
+int odr_implicit_settag(ODR o, int class, int tag);
+ </synopsis>
+ <para>
+ which overrides the tag of the type immediately following it. The
+ macro <function>odr_implicit_tag()</function> works by calling
+ <function>odr_implicit_settag()</function> immediately
+ before calling the function pointer argument.
+ Your type function could look like this:
+ </para>
+ <screen>
+int mySequence(ODR o, MySequence **p, int optional, const char *name)
+{
+ if (odr_implicit_settag(o, ODR_CONTEXT, 10) == 0 ||
+ odr_sequence_begin(o, p, sizeof(**p), name) == 0)
+ return optional && odr_ok(o);
+ return
+ odr_integer(o, &(*p)->intval, 0, "intval") &&
+ odr_bool(o, &(*p)->boolval, 1, "boolval") &&
+ odr_sequence_end(o);
+}
+ </screen>
+ <para>
+ The definition of the structure <literal>MySequence</literal> would be
+ the same.
+ </para>
+ </sect3>
+ <sect3 id="odr.explicit.tagging">
+ <title>Explicit Tagging</title>
+ <para>
+ Explicit tagging of constructed types is a little more complicated,
+ since you are in effect adding a level of construction to the data.
+ </para>
+ <para>
+ Assume the definition:
+ </para>
+ <screen>
+MySequence ::= [10] IMPLICIT SEQUENCE {
+ intval INTEGER,
+ boolval BOOLEAN OPTIONAL
+}
+ </screen>
+ <para>
+ Since the new type has an extra level of construction, two new functions
+ are needed to encapsulate the base type:
+ </para>
+ <synopsis>
+int odr_constructed_begin(ODR o, void *p, int class, int tag,
+ const char *name);
+
+int odr_constructed_end(ODR o);
+ </synopsis>
+ <para>
+ Assume that the IMPLICIT in the type definition above were replaced
+ with EXPLICIT (or that the IMPLICIT keyword were simply deleted, which
+ would be equivalent). The structure definition would look the same,
+ but the function would look like this:
+ </para>
+ <screen>
+int mySequence(ODR o, MySequence **p, int optional, const char *name)
+{
+ if (odr_constructed_begin(o, p, ODR_CONTEXT, 10, name) == 0)
+ return optional && odr_ok(o);
+ if (o->direction == ODR_DECODE)
+ *p = odr_malloc(o, sizeof(**p));
+ if (odr_sequence_begin(o, p, sizeof(**p), 0) == 0)
+ {
+ *p = 0; /* this is almost certainly a protocol error */
+ return 0;
+ }
+ return
+ odr_integer(o, &(*p)->intval, 0, "intval") &&
+ odr_bool(o, &(*p)->boolval, 1, "boolval") &&
+ odr_sequence_end(o) &&
+ odr_constructed_end(o);
+}
+ </screen>
+ <para>
+ Notice that the interface here gets kind of nasty. The reason is
+ simple: Explicitly tagged, constructed types are fairly rare in
+ the protocols that we care about, so the
+ esthetic annoyance (not to mention the dangers of a cluttered
+ interface) is less than the time that would be required to develop a
+ better interface. Nevertheless, it is far from satisfying, and it's a
+ point that will be worked on in the future. One option for you would
+ be to simply apply the <function>odr_explicit_tag()</function> macro to
+ the first function, and not
+ have to worry about <function>odr_constructed_*</function> yourself.
+ Incidentally, as you might have guessed, the
+ <function>odr_sequence_</function> functions are themselves
+ implemented using the <function>/odr_constructed_</function> functions.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2 id="odr.sequence.of">
+ <title>SEQUENCE OF</title>
+ <para>
+ To handle sequences (arrays) of a specific type, the function
+ </para>
+ <synopsis>
+int odr_sequence_of(ODR o, int (*fun)(ODR o, void *p, int optional),
+ void *p, int *num, const char *name);
+ </synopsis>
+ <para>
+ The <literal>fun</literal> parameter is a pointer to the decoder/encoder
+ function of the type. <literal>p</literal> is a pointer to an array of
+ pointers to your type. <literal>num</literal> is the number of elements
+ in the array.
+ </para>
+ <para>
+ Assume a type
+ </para>
+ <screen>
+MyArray ::= SEQUENCE OF INTEGER
+ </screen>
+ <para>
+ The C representation might be
+ </para>
+ <screen>
+typedef struct MyArray
+{
+ int num_elements;
+ Odr_int **elements;
+} MyArray;
+ </screen>
+ <para>
+ And the function might look like
+ </para>
+ <screen>
+int myArray(ODR o, MyArray **p, int optional, const char *name)
+{
+ if (o->direction == ODR_DECODE)
+ *p = odr_malloc(o, sizeof(**p));
+ if (odr_sequence_of(o, odr_integer, &(*p)->elements,
+ &(*p)->num_elements, name))
+ return 1;
+ *p = 0;
+ return optional && odr_ok(o);
+}
+ </screen>
+ </sect2>
+ <sect2 id="odr.choice.types">
+ <title>CHOICE Types</title>
+ <para>
+ The choice type is used fairly often in some ASN.1 definitions, so
+ some work has gone into streamlining its interface.
+ </para>
+ <para>
+ CHOICE types are handled by the function:
+ </para>
+ <synopsis>
+int odr_choice(ODR o, Odr_arm arm[], void *p, void *whichp,
+ const char *name);
+ </synopsis>
+ <para>
+ The <literal>arm</literal> array is used to describe each of the possible
+ types that the CHOICE type may assume. Internally in your application,
+ the CHOICE type is represented as a discriminated union. That is, a
+ C union accompanied by an integer (or enum) identifying the active
+ 'arm' of the union.
+ <literal>whichp</literal> is a pointer to the union discriminator.
+ When encoding, it is examined to determine the current type.
+ When decoding, it is set to reference the type that was found in
+ the input stream.
+ </para>
+ <para>
+ The Odr_arm type is defined thus:
+ </para>
+ <screen>
+typedef struct odr_arm
+{
+ int tagmode;
+ int class;
+ int tag;
+ int which;
+ Odr_fun fun;
+ char *name;
+} Odr_arm;
+ </screen>
+ <para>
+ The interpretation of the fields are:
+ </para>
+ <variablelist>
+ <varlistentry>
+ <term>tagmode</term>
+ <listitem><para>Either <literal>ODR_IMPLICIT</literal>,
+ <literal>ODR_EXPLICIT</literal>, or <literal>ODR_NONE</literal> (-1)
+ to mark no tagging.</para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>which</term>
+ <listitem><para>The value of the discriminator that corresponds to
+ this CHOICE element. Typically, it will be a #defined constant, or
+ an enum member.</para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>fun</term>
+ <listitem><para>A pointer to a function that implements the type of
+ the CHOICE member. It may be either a standard &odr; type or a type
+ defined by yourself.</para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>name</term>
+ <listitem><para>Name of tag.</para></listitem>
+ </varlistentry>
+ </variablelist>
+ <para>
+ A handy way to prepare the array for use by the
+ <function>odr_choice()</function> function is to
+ define it as a static, initialized array in the beginning of your
+ decoding/encoding function. Assume the type definition:
+ </para>
+ <screen>
+MyChoice ::= CHOICE {
+ untagged INTEGER,
+ tagged [99] IMPLICIT INTEGER,
+ other BOOLEAN
+}
+ </screen>
+ <para>
+ Your C type might look like
+ </para>
+ <screen>
+typedef struct MyChoice
+{
+ enum
+ {
+ MyChoice_untagged,
+ MyChoice_tagged,
+ MyChoice_other
+ } which;
+ union
+ {
+ Odr_int *untagged;
+ Odr_int *tagged;
+ Odr_bool *other;
+ } u;
+};
+ </screen>
+ <para>
+ And your function could look like this:
+ </para>
+ <screen>
+int myChoice(ODR o, MyChoice **p, int optional, const char *name)
+{
+ static Odr_arm arm[] =
+ {
+ {-1, -1, -1, MyChoice_untagged, odr_integer, "untagged"},
+ {ODR_IMPLICIT, ODR_CONTEXT, 99, MyChoice_tagged, odr_integer,
+ "tagged"},
+ {-1, -1, -1, MyChoice_other, odr_boolean, "other"},
+ {-1, -1, -1, -1, 0}
+ };
+
+ if (o->direction == ODR_DECODE)
+ *p = odr_malloc(o, sizeof(**p);
+ else if (!*p)
+ return optional && odr_ok(o);
+
+ if (odr_choice(o, arm, &(*p)->u, &(*p)->which), name)
+ return 1;
+ *p = 0;
+ return optional && odr_ok(o);
+}
+ </screen>
+ <para>
+ In some cases (say, a non-optional choice which is a member of a
+ sequence), you can "embed" the union and its discriminator in the
+ structure belonging to the enclosing type, and you won't need to
+ fiddle with memory allocation to create a separate structure to
+ wrap the discriminator and union.
+ </para>
+ <para>
+ The corresponding function is somewhat nicer in the Sun XDR interface.
+ Most of the complexity of this interface comes from the possibility of
+ declaring sequence elements (including CHOICEs) optional.
+ </para>
+ <para>
+ The ASN.1 specifications naturally requires that each member of a
+ CHOICE have a distinct tag, so they can be told apart on decoding.
+ Sometimes it can be useful to define a CHOICE that has multiple types
+ that share the same tag. You'll need some other mechanism, perhaps
+ keyed to the context of the CHOICE type. In effect, we would like to
+ introduce a level of context-sensitiveness to our ASN.1 specification.
+ When encoding an internal representation, we have no problem, as long
+ as each CHOICE member has a distinct discriminator value. For
+ decoding, we need a way to tell the choice function to look for a
+ specific arm of the table. The function
+ </para>
+ <synopsis>
+void odr_choice_bias(ODR o, int what);
+ </synopsis>
+ <para>
+ provides this functionality. When called, it leaves a notice for the next
+ call to <function>odr_choice()</function> to be called on the decoding
+ stream <literal>o</literal> that only the <literal>arm</literal> entry with
+ a <literal>which</literal> field equal to <literal>what</literal>
+ should be tried.
+ </para>
+ <para>
+ The most important application (perhaps the only one, really) is in
+ the definition of application-specific EXTERNAL encoders/decoders
+ which will automatically decode an ANY member given the direct or
+ indirect reference.
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="odr.debugging">
+ <title>Debugging</title>
+ <para>
+ The protocol modules are suffering somewhat from a lack of diagnostic
+ tools at the moment. Specifically ways to pretty-print PDUs that
+ aren't recognized by the system. We'll include something to this end
+ in a not-too-distant release. In the meantime, what we do when we get
+ packages we don't understand is to compile the ODR module with
+ <literal>ODR_DEBUG</literal> defined. This causes the module to dump tracing
+ information as it processes data units. With this output and the
+ protocol specification (Z39.50), it is generally fairly easy to see
+ what goes wrong.
+ </para>
+ </sect1>
+ </chapter>
+ <chapter id="comstack">
+ <title>The COMSTACK Module</title>
+ <sect1 id="comstack.synopsis">
+ <title>Synopsis (blocking mode)</title>
+ <programlisting><![CDATA[
+ COMSTACK stack;
+ char *buf = 0;
+ int size = 0, length_incoming;
+ char server_address_str[] = "localhost:9999";
+ void *server_address_ip;
+ int status;
+
+ char *protocol_package = "GET / HTTP/1.0\r\n\r\n";
+ int protocol_package_length = strlen(protocol_package);
+
+ stack = cs_create(tcpip_type, 1, PROTO_HTTP);
+ if (!stack) {
+ perror("cs_create"); /* use perror() here since we have no stack yet */
+ return -1;
+ }
+
+ server_address_ip = cs_straddr(stack, server_address_str);
+ if (!server_address_ip) {
+ fprintf(stderr, "cs_straddr: address could not be resolved\n");
+ return -1;
+ }
+
+ status = cs_connect(stack, server_address_ip);
+ if (status) {
+ fprintf(stderr, "cs_connect: %s\n", cs_strerror(stack));
+ return -1;
+ }
+
+ status = cs_rcvconnect(stack);
+ if (status) {
+ fprintf(stderr, "cs_rcvconnect: %s\n", cs_strerror(stack));
+ return -1;
+ }
+
+ status = cs_put(stack, protocol_package, protocol_package_length);
+ if (status) {
+ fprintf(stderr, "cs_put: %s\n", cs_strerror(stack));
+ return -1;
+ }
+
+ /* Now get a response */
+ length_incoming = cs_get(stack, &buf, &size);
+ if (!length_incoming) {
+ fprintf(stderr, "Connection closed\n");
+ return -1;
+ } else if (length_incoming < 0) {
+ fprintf(stderr, "cs_get: %s\n", cs_strerror(stack));
+ return -1;
+ }
+
+ /* Print result */
+ fwrite(buf, length_incoming, 1, stdout);
+
+ /* clean up */
+ cs_close(stack);
+ if (buf)
+ xfree(buf);
+ return 0;
+]]>
+ </programlisting>
+
+ </sect1>
+ <sect1 id="comstack.introduction">
+ <title>Introduction</title>
+ <para>
+ The &comstack;
+ subsystem provides a transparent interface to different types of transport
+ stacks for the exchange of BER-encoded data and HTTP packets.
+ At present, the RFC1729 method (BER over TCP/IP), local UNIX socket and an
+ experimental SSL stack are supported, but others may be added in time.
+ The philosophy of the
+ module is to provide a simple interface by hiding unused options and
+ facilities of the underlying libraries. This is always done at the risk
+ of losing generality, and it may prove that the interface will need
+ extension later on.
+ </para>
+ <note>
+ <para>
+ There hasn't been interest in the XTImOSI stack for some years.
+ Therefore, it is no longer supported.
+ </para>
+ </note>
+ <para>
+ The interface is implemented in such a fashion that only the
+ sub-layers constructed to the transport methods that you wish to
+ use in your application are linked in.
+ </para>
+ <para>
+ You will note that even though simplicity was a goal in the design,
+ the interface is still orders of magnitudes more complex than the
+ transport systems found in many other packages. One reason is that
+ the interface needs to support the somewhat different requirements of
+ the different lower-layer communications stacks; another important
+ reason is that the interface seeks to provide a more or less
+ industrial-strength approach to asynchronous event-handling.
+ When no function is allowed to block, things get more complex -
+ particularly on the server side.
+ We urge you to have a look at the demonstration client and server
+ provided with the package. They are meant to be easily readable and
+ instructive, while still being at least moderately useful.
+ </para>
+ </sect1>
+ <sect1 id="comstack.common">
+ <title>Common Functions</title>
+ <sect2 id="comstack.managing.endpoints">
+ <title>Managing Endpoints</title>
+ <synopsis>
+ COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
+ </synopsis>
+ <para>
+ Creates an instance of the protocol stack - a communications endpoint.
+ The <literal>type</literal> parameter determines the mode
+ of communication. At present the following values are supported:
+ </para>
+ <variablelist>
+ <varlistentry>
+ <term><literal>tcpip_type</literal></term>
+ <listitem><para>TCP/IP (BER over TCP/IP or HTTP over TCP/IP)
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>ssl_type</literal></term>
+ <listitem><para>Secure Socket Layer (SSL). This COMSTACK
+ is experimental and is not fully implemented. If
+ HTTP is used, this effectively is HTTPS.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>unix_type</literal></term>
+ <listitem><para>Unix socket (unix only). Local Transfer via
+ file socket. See <citerefentry><refentrytitle>unix</refentrytitle>
+ <manvolnum>7</manvolnum></citerefentry>.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ <para>
+ The <function>cs_create</function> function returns a null-pointer
+ if a system error occurs.
+ The <literal>blocking</literal> parameter should be one if
+ you wish the association to operate in blocking mode, zero otherwise.
+ The <literal>protocol</literal> field should be
+ <literal>PROTO_Z3950</literal> or <literal>PROTO_HTTP</literal>.
+ Protocol <literal>PROTO_SR</literal> is no longer supported.
+ </para>
+ <synopsis>
+ void cs_close(COMSTACK handle);
+ </synopsis>
+ <para>
+ Closes the connection (as elegantly as the lower layers will permit),
+ and releases the resources pointed to by the
+ <literal>handle</literal>
+ parameter. The
+ <literal>handle</literal>
+ should not be referenced again after this call.
+ </para>
+ <note>
+ <para>
+ We really need a soft disconnect, don't we?
+ </para>
+ </note>
+ </sect2>
+ <sect2 id="comstack.data.exchange">
+ <title>Data Exchange</title>
+ <synopsis>
+ int cs_put(COMSTACK handle, char *buf, int len);
+ </synopsis>
+ <para>
+ Sends <literal>buf</literal> down the wire.
+ In blocking mode, this function will return only when a full buffer has
+ been written, or an error has occurred. In nonblocking mode, it's
+ possible that the function will be unable to send the full buffer
+ at once, which will be indicated by a return value of 1.
+ The function will keep track of the number of octets already written; you
+ should call it repeatedly with the same values of <literal>buf</literal>
+ and <literal>len</literal>, until the buffer has been transmitted.
+ When a full buffer has been sent, the function will return 0 for
+ success. -1 indicates an error condition (see below).
+ </para>
+ <synopsis>
+ int cs_get(COMSTACK handle, char **buf, int *size);
+ </synopsis>
+ <para>
+ Receives a PDU or HTTP Response from the peer. Returns the number of
+ bytes read.
+ In nonblocking mode, it is possible that not all of the packet can be
+ read at once. In this case, the function returns 1. To simplify the
+ interface, the function is
+ responsible for managing the size of the buffer. It will be reallocated
+ if necessary to contain large packages, and will sometimes be moved
+ around internally by the subsystem when partial packages are read. Before
+ calling
+ <function>cs_get</function>
+ for the fist time, the buffer can be initialized to the null pointer,
+ and the length should also be set to 0 - cs_get will perform a
+ <function>malloc(2)</function>
+ on the buffer for you. When a full buffer has been read, the size of
+ the package is returned (which will always be greater than 1). -1
+ indicates an error condition.
+ </para>
+ <para>
+ See also the <function>cs_more()</function> function below.
+ </para>
+ <synopsis>
+ int cs_more(COMSTACK handle);
+ </synopsis>
+ <para>
+ The <function>cs_more()</function> function should be used in conjunction
+ with <function>cs_get</function> and
+ <function>select(2)</function>.
+ The <function>cs_get()</function> function will sometimes
+ (notably in the TCP/IP mode) read more than a single protocol package
+ off the network. When this happens, the extra package is stored
+ by the subsystem. After calling <function>cs_get()</function>, and before
+ waiting for more input, You should always call
+ <function>cs_more()</function>
+ to check if there's a full protocol package already read. If
+ <function>cs_more()</function>
+ returns 1,
+ <function>cs_get()</function>
+ can be used to immediately fetch the new package. For the
+ mOSI
+ subsystem, the function should always return 0, but if you want your
+ stuff to be protocol independent, you should use it.
+ </para>
+ <note>
+ <para>
+ The <function>cs_more()</function>
+ function is required because the RFC1729-method
+ does not provide a way of separating individual PDUs, short of
+ partially decoding the BER. Some other implementations will carefully
+ nibble at the packet by calling
+ <function>read(2)</function>
+ several times. This was felt to be too inefficient (or at least
+ clumsy) - hence the call for this extra function.
+ </para>
+ </note>
+ <synopsis>
+ int cs_look(COMSTACK handle);
+ </synopsis>
+ <para>
+ This function is useful when you're operating in nonblocking
+ mode. Call it when
+ <function>select(2)</function>
+ tells you there's something happening on the line. It returns one of
+ the following values:
+ </para>
+ <variablelist>
+ <varlistentry>
+ <term>CS_NONE</term>
+ <listitem><para>
+ No event is pending. The data found on the line was not a
+ complete package.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>CS_CONNECT</term>
+ <listitem><para>
+ A response to your connect request has been received. Call
+ <function>cs_rcvconnect</function>
+ to process the event and to finalize the connection establishment.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>CS_DISCON</term>
+ <listitem><para>
+ The other side has closed the connection (or maybe sent a disconnect
+ request - but do we care? Maybe later). Call
+ <function>cs_close</function> to close your end of the association
+ as well.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>CS_LISTEN</term>
+ <listitem><para>
+ A connect request has been received.
+ Call <function>cs_listen</function> to process the event.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term>CS_DATA</term>
+ <listitem><para>
+ There's data to be found on the line.
+ Call <function>cs_get</function> to get it.
+ </para></listitem>
+ </varlistentry>
+ </variablelist>
+ <note>
+ <para>
+ You should be aware that even if
+ <function>cs_look()</function>
+ tells you that there's an event event pending, the corresponding
+ function may still return and tell you there was nothing to be found.
+ This means that only part of a package was available for reading. The
+ same event will show up again, when more data has arrived.
+ </para>
+ </note>
+ <synopsis>
+ int cs_fileno(COMSTACK h);
+ </synopsis>
+ <para>
+ Returns the file descriptor of the association. Use this when
+ file-level operations on the endpoint are required
+ (<function>select(2)</function> operations, specifically).
+ </para>
+ </sect2>
+ </sect1>
+ <sect1 id="comstack.client">
+ <title>Client Side</title>
+ <synopsis>
+ int cs_connect(COMSTACK handle, void *address);
+ </synopsis>
+ <para>
+ Initiate a connection with the target at <literal>address</literal>
+ (more on addresses below). The function will return 0 on success, and 1 if
+ the operation does not complete immediately (this will only
+ happen on a nonblocking endpoint). In this case, use
+ <function>cs_rcvconnect</function> to complete the operation,
+ when <function>select(2)</function> or <function>poll(2)</function>
+ reports input pending on the association.
+ </para>
+ <synopsis>
+ int cs_rcvconnect(COMSTACK handle);
+ </synopsis>
+ <para>
+ Complete a connect operation initiated by <function>cs_connect()</function>.
+ It will return 0 on success; 1 if the operation has not yet completed (in
+ this case, call the function again later); -1 if an error has occurred.
+ </para>
+ </sect1>
+ <sect1 id="comstack.server">
+ <title>Server Side</title>
+ <para>
+ To establish a server under the <application>inetd</application>
+ server, you can use
+ </para>
+ <synopsis>
+ COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking,
+ int protocol);
+ </synopsis>
+ <para>
+ The <literal>socket</literal> parameter is an established socket (when
+ your application is invoked from <application>inetd</application>, the
+ socket will typically be 0.
+ The following parameters are identical to the ones for
+ <function>cs_create</function>.
+ </para>
+ <synopsis>
+ int cs_bind(COMSTACK handle, void *address, int mode)
+ </synopsis>
+ <para>
+ Binds a local address to the endpoint. Read about addresses below. The
+ <literal>mode</literal> parameter should be either
+ <literal>CS_CLIENT</literal> or <literal>CS_SERVER</literal>.
+ </para>
+ <synopsis>
+ int cs_listen(COMSTACK handle, char *addr, int *addrlen);
+ </synopsis>
+ <para>
+ Call this to process incoming events on an endpoint that has been
+ bound in listening mode. It will return 0 to indicate that the connect
+ request has been received, 1 to signal a partial reception, and -1 to
+ indicate an error condition.
+ </para>
+ <synopsis>
+ COMSTACK cs_accept(COMSTACK handle);
+ </synopsis>
+ <para>
+ This finalizes the server-side association establishment, after
+ cs_listen has completed successfully. It returns a new connection
+ endpoint, which represents the new association. The application will
+ typically wish to fork off a process to handle the association at this
+ point, and continue listen for new connections on the old
+ <literal>handle</literal>.
+ </para>
+ <para>
+ You can use the call
+ </para>
+ <synopsis>
+ const char *cs_addrstr(COMSTACK);
+ </synopsis>
+ <para>
+ on an established connection to retrieve the host-name of the remote host.
+ </para>
+ <note>
+ <para>
+ You may need to use this function with some care if your
+ name server service is slow or unreliable
+ </para>
+ </note>
+ </sect1>
+ <sect1 id="comstack.addresses">
+ <title>Addresses</title>
+ <para>
+ The low-level format of the addresses are different depending on the
+ mode of communication you have chosen. A function is provided by each
+ of the lower layers to map a user-friendly string-form address to the
+ binary form required by the lower layers.
+ </para>
+ <synopsis>
+ void *cs_straddr(COMSTACK handle, const char *str);
+ </synopsis>
+ <para>
+ The format for TCP/IP and SSL addresses is:
+ </para>
+ <synopsis>
+ <host> [ ':' <portnum> ]
+ </synopsis>
+ <para>
+ The <literal>hostname</literal> can be either a domain name or an
+ IP address. The port number, if omitted, defaults to 210.
+ </para>
+ <para>
+ For TCP/IP and SSL, the special hostnames <literal>@</literal>,
+ maps to <literal>IN6ADDR_ANY_INIT</literal> with
+ IPV4 binding as well (bindv6only=0),
+ The special hostname <literal>@4</literal> binds to
+ <literal>INADDR_ANY</literal> (IPV4 only listener).
+ The special hostname <literal>@6</literal> binds to
+ <literal>IN6ADDR_ANY_INIT</literal> with bindv6only=1 (IPV6 only listener).
+ </para>
+ <para>
+ For UNIX sockets, the format of an address is the socket filename.
+ </para>
+ <para>
+ When a connection has been established, you can use
+ </para>
+ <synopsis>
+ const char *cs_addrstr(COMSTACK h);
+ </synopsis>
+ <para>
+ to retrieve the host name of the peer system. The function returns
+ a pointer to a static area, which is overwritten on the next call
+ to the function.
+ </para>
+ <para>
+ A fairly recent addition to the &comstack; module is the utility
+ function
+ </para>
+ <synopsis>
+ COMSTACK cs_create_host (const char *str, int blocking, void **vp);
+ </synopsis>
+ <para>
+ which is just a wrapper for <function>cs_create</function> and
+ <function>cs_straddr</function>. The <parameter>str</parameter>
+ is similar to that described for <function>cs_straddr</function>
+ but with a prefix denoting the &comstack; type. Prefixes supported
+ are <literal>tcp:</literal>, <literal>unix:</literal> and
+ <literal>ssl:</literal> for TCP/IP, UNIX and SSL respectively.
+ If no prefix is given, then TCP/IP is used.
+ The <parameter>blocking</parameter> is passed to
+ function <function>cs_create</function>. The third parameter
+ <parameter>vp</parameter> is a pointer to &comstack; stack type
+ specific values.
+ Parameter <parameter>vp</parameter> is reserved for future use.
+ Set it to <literal>NULL</literal>.
+ </para>
+ </sect1>
+ <sect1 id="comstack.ssl">
+ <title>SSL</title>
+ <para>
+ <synopsis>
+ void *cs_get_ssl(COMSTACK cs);
+ </synopsis>
+ Returns the SSL handle, <literal>SSL *</literal> for comstack. If comstack
+ is not of type SSL, NULL is returned.
+ </para>
+ <para>
+ <synopsis>
+ int cs_set_ssl_ctx(COMSTACK cs, void *ctx);
+ </synopsis>
+ Sets SSL context for comstack. The parameter is expected to be of type
+ <literal>SSL_CTX *</literal>. This function should be called just
+ after comstack has been created (before connect, bind, etc).
+ This function returns 1 for success; 0 for failure.
+ </para>
+ <para>
+ <synopsis>
+ int cs_set_ssl_certificate_file(COMSTACK cs, const char *fname);
+ </synopsis>
+ Sets SSL certificate for comstack as a PEM file. This function
+ returns 1 for success; 0 for failure.
+ </para>
+ <para>
+ <synopsis>
+ int cs_get_ssl_peer_certificate_x509(COMSTACK cs, char **buf, int *len);
+ </synopsis>
+ This function returns the peer certificate. If successful,
+ <literal>*buf</literal> and <literal>*len</literal> holds
+ X509 buffer and length respectively. Buffer should be freed
+ with <literal>xfree</literal>. This function returns 1 for success;
+ 0 for failure.
+ </para>
+ </sect1>
+ <sect1 id="comstack.diagnostics">
+ <title>Diagnostics</title>
+ <para>
+ All functions return -1 if an error occurs. Typically, the functions
+ will return 0 on success, but the data exchange functions
+ (<function>cs_get</function>, <function>cs_put</function>,
+ <function>cs_more</function>) follow special rules. Consult their
+ descriptions.
+ </para>
+ <para>
+ The error code for the COMSTACK can be retrieved using C macro
+ <function>cs_errno</function> which will return one
+ of the error codes <literal>CSYSERR</literal>,
+ <literal>CSOUTSTATE</literal>,
+ <literal>CSNODATA</literal>, ...
+ </para>
+ <synopsis>
+ int cs_errno(COMSTACK handle);
+ </synopsis>
+ <para>
+ You can the textual representation of the error code
+ by using <function>cs_errmsg</function> - which
+ works like <function>strerror(3)</function>
+ </para>
+ <synopsis>
+ const char *cs_errmsg(int n);
+ </synopsis>
+ <para>
+ It is also possible to get straight to the textual represenataion
+ without the error code by using
+ <function>cs_strerror</function>.
+ </para>
+ <synopsis>
+ const char *cs_strerror(COMSTACK h);
+ </synopsis>
+ </sect1>
+ <sect1 id="comstack.summary">
+ <title>Summary and Synopsis</title>
+ <synopsis><![CDATA[
+ #include <yaz/comstack.h>
+
+ #include <yaz/tcpip.h> /* this is for TCP/IP and SSL support */
+ #include <yaz/unix.h> /* this is for UNIX socket support */
+
+ COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
+
+ COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking,
+ int protocol);
+ COMSTACK cs_create_host(const char *str, int blocking,
+ void **vp);
+
+ int cs_bind(COMSTACK handle, int mode);
+
+ int cs_connect(COMSTACK handle, void *address);
+
+ int cs_rcvconnect(COMSTACK handle);
+
+ int cs_listen(COMSTACK handle);
+
+ COMSTACK cs_accept(COMSTACK handle);
+
+ int cs_put(COMSTACK handle, char *buf, int len);
+
+ int cs_get(COMSTACK handle, char **buf, int *size);
+
+ int cs_more(COMSTACK handle);
+
+ void cs_close(COMSTACK handle);
+
+ int cs_look(COMSTACK handle);
+
+ void *cs_straddr(COMSTACK handle, const char *str);
+
+ const char *cs_addrstr(COMSTACK h);
+]]>
+ </synopsis>
+ </sect1>
+ </chapter>
+ <chapter id="future">
+ <title>Future Directions</title>
+ <para>
+ We have a new and better version of the front-end server on the drawing
+ board. Resources and external commitments will govern when we'll be
+ able to do something real with it. Features should include greater
+ flexibility, greater support for access/resource control, and easy
+ support for Explain (possibly with Zebra as an extra database engine).
+ </para>
+ <para>
+ &yaz; is a BER toolkit and as such should support all protocols
+ out there based on that. We'd like to see running ILL applications.
+ It shouldn't be that hard. Another thing that would be interesting is
+ LDAP. Maybe a generic framework for doing IR using both LDAP and
+ Z39.50 transparently.
+ </para>
+ <para>
+ The SOAP implementation is incomplete. In the future we hope
+ to add more features to it. Perhaps make a WSDL/XML Schema compiler.
+ The authors of libxml2 are already working on XML Schema / RelaxNG
+ compilers so this may not be too hard.
+ </para>
+ <para>
+ It would be neat to have a proper module mechanism for the Generic
+ Frontend Server so that backend would be dynamically
+ loaded (as shared objects / DLLs).
+ </para>
+ <para>
+ Other than that, &yaz; generally moves in the directions which appear to
+ make the most people happy (including ourselves, as prime users of the
+ software). If there's something you'd like to see in here, then drop
+ us a note and let's see what we can come up with.
+ </para>
+ </chapter>
+ <reference id="reference">
+ <title>Reference</title>
+ <partintro id="reference-introduction">
+ <para>
+ The material in this chapter is drawn directly from the individual
+ manual entries.
+ </para>
+ </partintro>
+ &manref;
+ </reference>
+ <appendix id="list-oids">
+ <title>List of Object Identifiers</title>
+ <para>
+ These is a list of object identifiers that are built into YAZ.
+ </para>
+ &std-oid-table;
+ </appendix>
+ <appendix id="bib1-diagnostics">
+ <title>Bib-1 diagnostics</title>
+ <para>
+ List of Bib-1 diagnostics that are known to YAZ.
+ </para>
+ &bib1-diag-table;
+ </appendix>
+ <appendix id="sru-diagnostics">
+ <title>SRU diagnostics</title>
+ <para>
+ List of SRU diagnostics that are known to YAZ.
+ </para>
+ &srw-diag-table;
+ </appendix>
+ <appendix id="license">
+ <title>License</title>
+ <sect1 id="license.indexdata">
+ <title>Index Data Copyright</title>
+ <para>
+ Copyright © ©right-year; Index Data.
+ </para>
+ <para>
+ All rights reserved.
+ </para>
+ <para>
+ Redistribution and use in source and binary forms, with or without
+ modification, are permitted provided that the following conditions are met:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Redistributions in binary form must reproduce the above copyright
+ notice, this list of conditions and the following disclaimer in the
+ documentation and/or other materials provided with the distribution.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Neither the name of Index Data nor the names of its contributors
+ may be used to endorse or promote products derived from this
+ software without specific prior written permission.
+ </para>
+ </listitem>
+ </itemizedlist>
+ <para>
+ THIS SOFTWARE IS PROVIDED BY INDEX DATA ``AS IS'' AND ANY
+ EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ DISCLAIMED. IN NO EVENT SHALL INDEX DATA BE LIABLE FOR
+ ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ SUCH DAMAGE.
+ </para>
+ </sect1>
+ </appendix>
+ <appendix id="indexdata">
+ <title>About Index Data</title>
+ <para>
+ Index Data is a consulting and software-development enterprise that
+ specializes in library and information management systems. Our
+ interests and expertise span a broad range of related fields, and one
+ of our primary, long-term objectives is the development of a powerful
+ information management
+ system with open network interfaces and hyper-media capabilities.
+ </para><para>
+ We make this software available free of charge, on a fairly unrestrictive
+ license; as a service to the networking community, and to further the
+ development of quality software for open network communication.
+ </para><para>
+ We'll be happy to answer questions about the software, and about ourselves
+ in general.
+ </para>
+ <para>
+ <address>
+ Index Data ApS
+ <street>Amagerfælledvej 56</street>
+ <postcode>2300 Copenhagen S</postcode>
+ <country>Denmark</country>
+ Email <email>info@indexdata.dk</email>
+ </address>
+ </para>
+ <para>
+ The Hacker's Jargon File has the following to say about the
+ use of the
+ prefix "YA" in the name of a software product.
+ </para>
+ <para>
+ <citation>
+ Yet Another. adj. 1. Of your own work: A
+ humorous allusion often used in titles to acknowledge that the
+ topic is not original, though the content is. As in "Yet Another
+ AI Group" or "Yet Another Simulated Annealing Algorithm".
+ 2. Of
+ others' work: Describes something of which there are already far
+ too many.
+ </citation>
+ </para>
+ </appendix>
+ <appendix id="credits">
+ <title>Credits</title>
+ <para>
+ This appendix lists individuals that have contributed in the development
+ of &yaz;. Some have contributed with code, while others have provided bug
+ fixes or suggestions. If we're missing somebody, of if you, for
+ whatever reason, don't like to be listed here, let us know.
+ </para>
+ <itemizedlist>
+ <listitem><para>
+ Gary Anderson
+ </para></listitem>
+ <listitem><para>
+ Dimitrios Andreadis
+ </para></listitem>
+ <listitem><para>
+ Morten Bøgeskov
+ </para></listitem>
+ <listitem><para>
+ Rocco Carbone
+ </para></listitem>
+ <listitem><para>
+ Matthew Carey
+ </para></listitem>
+ <listitem><para>
+ Hans van Dalen
+ </para></listitem>
+ <listitem><para>
+ Irina Dijour
+ </para></listitem>
+ <listitem><para>
+ Larry E. Dixson
+ </para></listitem>
+ <listitem><para>
+ Hans van den Dool
+ </para></listitem>
+ <listitem><para>
+ Mads Bondo Dydensborg
+ </para></listitem>
+ <listitem><para>
+ Franck Falcoz
+ </para></listitem>
+ <listitem><para>
+ Kevin Gamiel
+ </para></listitem>
+ <listitem><para>
+ Morten Garkier Hendriksen
+ </para></listitem>
+ <listitem><para>
+ Morten Holmqvist
+ </para></listitem>
+ <listitem><para>
+ Ian Ibbotson
+ </para></listitem>
+ <listitem><para>
+ Shigeru Ishida
+ </para></listitem>
+ <listitem><para>
+ Heiko Jansen
+ </para></listitem>
+ <listitem><para>
+ David Johnson
+ </para></listitem>
+ <listitem><para>
+ Oleg Kolobov
+ </para></listitem>
+ <listitem><para>
+ Giannis Kosmas
+ </para></listitem>
+ <listitem><para>
+ Kang-Jin Lee
+ </para></listitem>
+ <listitem><para>
+ Pieter Van Lierop
+ </para></listitem>
+ <listitem><para>
+ Stefan Lohrum
+ </para></listitem>
+ <listitem><para>
+ Ronald van der Meer
+ </para></listitem>
+ <listitem><para>
+ Thomas W. Place
+ </para></listitem>
+ <listitem><para>
+ Peter Popovics
+ </para></listitem>
+ <listitem><para>
+ Jacob Chr. Poulsen
+ </para></listitem>
+ <listitem><para>
+ Ko van der Sloot
+ </para></listitem>
+ <listitem><para>
+ Mike Taylor
+ </para></listitem>
+ <listitem><para>
+ Rustam T. Usmanov
+ </para></listitem>
+ <listitem><para>
+ Charles Woodfield
+ </para></listitem>
+ <listitem><para>
+ Tom André Øverland
+ </para></listitem>
+ </itemizedlist>
+ </appendix>
+</book>
+
+<!-- Keep this comment at the end of the file
+Local variables:
+mode: nxml
+nxml-child-indent: 1
+End:
+-->
+++ /dev/null
- <bookinfo>
- <title>YAZ User's Guide and Reference</title>
- <authorgroup>
- <author><firstname>Sebastian</firstname><surname>Hammer</surname></author>
- <author><firstname>Adam</firstname><surname>Dickmeiss</surname></author>
- <author><firstname>Mike</firstname><surname>Taylor</surname></author>
- <author><firstname>Heikki</firstname><surname>Levanto</surname></author>
- <author><firstname>Dennis</firstname><surname>Schafroth</surname></author>
- </authorgroup>
- <releaseinfo>&version;</releaseinfo>
- <copyright>
- <year>©right-year;</year>
- <holder>Index Data</holder>
- </copyright>
- <abstract><simpara>
- This document is the programmer's guide and reference to the &yaz;
- package version &version;. &yaz; is a compact toolkit that provides
- access to the Z39.50 and SRU/Solr protocols, as well as a set of
- higher-level tools for implementing the server and client
- roles, respectively.
- The documentation can be used on its own, or as a reference when
- looking at the example applications provided with the package.
- </simpara>
- <simpara>
- <inlinemediaobject>
- <imageobject>
- <imagedata fileref="common/id.png" format="PNG"/>
- </imageobject>
- <imageobject>
- <imagedata fileref="common/id.eps" format="EPS"/>
- </imageobject>
- </inlinemediaobject>
- </simpara></abstract>
- </bookinfo>
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
+++ /dev/null
- <chapter id="comstack"><title>The COMSTACK Module</title>
-
- <sect1 id="comstack.synopsis"><title>Synopsis (blocking mode)</title>
-
- <programlisting><![CDATA[
- COMSTACK stack;
- char *buf = 0;
- int size = 0, length_incoming;
- char server_address_str[] = "localhost:9999";
- void *server_address_ip;
- int status;
-
- char *protocol_package = "GET / HTTP/1.0\r\n\r\n";
- int protocol_package_length = strlen(protocol_package);
-
- stack = cs_create(tcpip_type, 1, PROTO_HTTP);
- if (!stack) {
- perror("cs_create"); /* use perror() here since we have no stack yet */
- return -1;
- }
-
- server_address_ip = cs_straddr(stack, server_address_str);
- if (!server_address_ip) {
- fprintf(stderr, "cs_straddr: address could not be resolved\n");
- return -1;
- }
-
- status = cs_connect(stack, server_address_ip);
- if (status) {
- fprintf(stderr, "cs_connect: %s\n", cs_strerror(stack));
- return -1;
- }
-
- status = cs_rcvconnect(stack);
- if (status) {
- fprintf(stderr, "cs_rcvconnect: %s\n", cs_strerror(stack));
- return -1;
- }
-
- status = cs_put(stack, protocol_package, protocol_package_length);
- if (status) {
- fprintf(stderr, "cs_put: %s\n", cs_strerror(stack));
- return -1;
- }
-
- /* Now get a response */
- length_incoming = cs_get(stack, &buf, &size);
- if (!length_incoming) {
- fprintf(stderr, "Connection closed\n");
- return -1;
- } else if (length_incoming < 0) {
- fprintf(stderr, "cs_get: %s\n", cs_strerror(stack));
- return -1;
- }
-
- /* Print result */
- fwrite(buf, length_incoming, 1, stdout);
-
- /* clean up */
- cs_close(stack);
- if (buf)
- xfree(buf);
- return 0;
-]]>
- </programlisting>
-
- </sect1>
- <sect1 id="comstack.introduction"><title>Introduction</title>
-
- <para>
- The &comstack;
- subsystem provides a transparent interface to different types of transport
- stacks for the exchange of BER-encoded data and HTTP packets.
- At present, the RFC1729 method (BER over TCP/IP), local UNIX socket and an
- experimental SSL stack are supported, but others may be added in time.
- The philosophy of the
- module is to provide a simple interface by hiding unused options and
- facilities of the underlying libraries. This is always done at the risk
- of losing generality, and it may prove that the interface will need
- extension later on.
- </para>
-
- <note>
- <para>
- There hasn't been interest in the XTImOSI stack for some years.
- Therefore, it is no longer supported.
- </para>
- </note>
-
- <para>
- The interface is implemented in such a fashion that only the
- sub-layers constructed to the transport methods that you wish to
- use in your application are linked in.
- </para>
-
- <para>
- You will note that even though simplicity was a goal in the design,
- the interface is still orders of magnitudes more complex than the
- transport systems found in many other packages. One reason is that
- the interface needs to support the somewhat different requirements of
- the different lower-layer communications stacks; another important
- reason is that the interface seeks to provide a more or less
- industrial-strength approach to asynchronous event-handling.
- When no function is allowed to block, things get more complex -
- particularly on the server side.
- We urge you to have a look at the demonstration client and server
- provided with the package. They are meant to be easily readable and
- instructive, while still being at least moderately useful.
- </para>
-
- </sect1>
- <sect1 id="comstack.common"><title>Common Functions</title>
-
- <sect2 id="comstack.managing.endpoints"><title>Managing Endpoints</title>
-
- <synopsis>
- COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
- </synopsis>
-
- <para>
- Creates an instance of the protocol stack - a communications endpoint.
- The <literal>type</literal> parameter determines the mode
- of communication. At present the following values are supported:
- </para>
-
- <variablelist>
- <varlistentry><term><literal>tcpip_type</literal></term>
- <listitem><para>TCP/IP (BER over TCP/IP or HTTP over TCP/IP)
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>ssl_type</literal></term>
- <listitem><para>Secure Socket Layer (SSL). This COMSTACK
- is experimental and is not fully implemented. If
- HTTP is used, this effectively is HTTPS.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>unix_type</literal></term>
- <listitem><para>Unix socket (unix only). Local Transfer via
- file socket. See <citerefentry><refentrytitle>unix</refentrytitle>
- <manvolnum>7</manvolnum></citerefentry>.
- </para></listitem>
- </varlistentry>
- </variablelist>
-
- <para>
- The <function>cs_create</function> function returns a null-pointer
- if a system error occurs.
- The <literal>blocking</literal> parameter should be one if
- you wish the association to operate in blocking mode, zero otherwise.
- The <literal>protocol</literal> field should be
- <literal>PROTO_Z3950</literal> or <literal>PROTO_HTTP</literal>.
- Protocol <literal>PROTO_SR</literal> is no longer supported.
- </para>
-
- <synopsis>
- void cs_close(COMSTACK handle);
- </synopsis>
-
- <para>
- Closes the connection (as elegantly as the lower layers will permit),
- and releases the resources pointed to by the
- <literal>handle</literal>
- parameter. The
- <literal>handle</literal>
- should not be referenced again after this call.
- </para>
-
- <note>
- <para>
- We really need a soft disconnect, don't we?
- </para>
- </note>
- </sect2>
-
- <sect2 id="comstack.data.exchange"><title>Data Exchange</title>
-
- <synopsis>
- int cs_put(COMSTACK handle, char *buf, int len);
- </synopsis>
-
- <para>
- Sends
- <literal>buf</literal>
- down the wire. In blocking mode, this function will return only when a
- full buffer has been written, or an error has occurred. In nonblocking
- mode, it's possible that the function will be unable to send the full
- buffer at once, which will be indicated by a return value of 1. The
- function will keep track of the number of octets already written; you
- should call it repeatedly with the same values of <literal>buf</literal>
- and <literal>len</literal>, until the buffer has been transmitted.
- When a full buffer has been sent, the function will return 0 for
- success. -1 indicates an error condition (see below).
- </para>
-
- <synopsis>
- int cs_get(COMSTACK handle, char **buf, int *size);
- </synopsis>
-
- <para>
- Receives a PDU or HTTP Response from the peer. Returns the number of
- bytes read.
- In nonblocking mode, it is possible that not all of the packet can be
- read at once. In this case, the function returns 1. To simplify the
- interface, the function is
- responsible for managing the size of the buffer. It will be reallocated
- if necessary to contain large packages, and will sometimes be moved
- around internally by the subsystem when partial packages are read. Before
- calling
- <function>cs_get</function>
- for the fist time, the buffer can be initialized to the null pointer,
- and the length should also be set to 0 - cs_get will perform a
- <function>malloc(2)</function>
- on the buffer for you. When a full buffer has been read, the size of
- the package is returned (which will always be greater than 1). -1
- indicates an error condition.
- </para>
-
- <para>
- See also the <function>cs_more()</function> function below.
- </para>
-
- <synopsis>
- int cs_more(COMSTACK handle);
- </synopsis>
-
- <para>
- The <function>cs_more()</function> function should be used in conjunction
- with <function>cs_get</function> and
- <function>select(2)</function>.
- The <function>cs_get()</function> function will sometimes
- (notably in the TCP/IP mode) read more than a single protocol package
- off the network. When this happens, the extra package is stored
- by the subsystem. After calling <function>cs_get()</function>, and before
- waiting for more input, You should always call
- <function>cs_more()</function>
- to check if there's a full protocol package already read. If
- <function>cs_more()</function>
- returns 1,
- <function>cs_get()</function>
- can be used to immediately fetch the new package. For the
- mOSI
- subsystem, the function should always return 0, but if you want your
- stuff to be protocol independent, you should use it.
- </para>
-
- <note>
- <para>
- The <function>cs_more()</function>
- function is required because the RFC1729-method
- does not provide a way of separating individual PDUs, short of
- partially decoding the BER. Some other implementations will carefully
- nibble at the packet by calling
- <function>read(2)</function>
- several times. This was felt to be too inefficient (or at least
- clumsy) - hence the call for this extra function.
- </para>
- </note>
-
- <synopsis>
- int cs_look(COMSTACK handle);
- </synopsis>
-
- <para>
- This function is useful when you're operating in nonblocking
- mode. Call it when
- <function>select(2)</function>
- tells you there's something happening on the line. It returns one of
- the following values:
- </para>
-
- <variablelist>
- <varlistentry><term>CS_NONE</term><listitem><para>
- No event is pending. The data found on the line was not a
- complete package.
- </para></listitem></varlistentry>
-
- <varlistentry><term>CS_CONNECT</term><listitem><para>
- A response to your connect request has been received. Call
- <function>cs_rcvconnect</function>
- to process the event and to finalize the connection establishment.
- </para></listitem></varlistentry>
-
- <varlistentry><term>CS_DISCON</term><listitem><para>
- The other side has closed the connection (or maybe sent a disconnect
- request - but do we care? Maybe later). Call
- <function>cs_close</function> to close your end of the association
- as well.
- </para></listitem></varlistentry>
-
- <varlistentry><term>CS_LISTEN</term><listitem><para>
- A connect request has been received.
- Call <function>cs_listen</function> to process the event.
- </para></listitem></varlistentry>
-
- <varlistentry><term>CS_DATA</term><listitem><para>
- There's data to be found on the line.
- Call <function>cs_get</function> to get it.
- </para></listitem></varlistentry>
- </variablelist>
-
- <note>
- <para>
- You should be aware that even if
- <function>cs_look()</function>
- tells you that there's an event event pending, the corresponding
- function may still return and tell you there was nothing to be found.
- This means that only part of a package was available for reading. The
- same event will show up again, when more data has arrived.
- </para>
- </note>
-
- <synopsis>
- int cs_fileno(COMSTACK h);
- </synopsis>
-
- <para>
- Returns the file descriptor of the association. Use this when
- file-level operations on the endpoint are required
- (<function>select(2)</function> operations, specifically).
- </para>
- </sect2>
-
- </sect1>
-
- <sect1 id="comstack.client"><title>Client Side</title>
-
- <synopsis>
- int cs_connect(COMSTACK handle, void *address);
- </synopsis>
-
- <para>
- Initiate a connection with the target at <literal>address</literal>
- (more on addresses below). The function will return 0 on success, and 1 if
- the operation does not complete immediately (this will only
- happen on a nonblocking endpoint). In this case, use
- <function>cs_rcvconnect</function> to complete the operation,
- when <function>select(2)</function> or <function>poll(2)</function>
- reports input pending on the association.
- </para>
-
- <synopsis>
- int cs_rcvconnect(COMSTACK handle);
- </synopsis>
-
- <para>
- Complete a connect operation initiated by <function>cs_connect()</function>.
- It will return 0 on success; 1 if the operation has not yet completed (in
- this case, call the function again later); -1 if an error has occurred.
- </para>
-
- </sect1>
-
- <sect1 id="comstack.server"><title>Server Side</title>
-
- <para>
- To establish a server under the <application>inetd</application>
- server, you can use
- </para>
-
- <synopsis>
- COMSTACK cs_createbysocket(int socket, CS_TYPE type, int blocking,
- int protocol);
- </synopsis>
-
- <para>
- The <literal>socket</literal> parameter is an established socket (when
- your application is invoked from <application>inetd</application>, the
- socket will typically be 0.
- The following parameters are identical to the ones for
- <function>cs_create</function>.
- </para>
-
- <synopsis>
- int cs_bind(COMSTACK handle, void *address, int mode)
- </synopsis>
-
- <para>
- Binds a local address to the endpoint. Read about addresses below. The
- <literal>mode</literal> parameter should be either
- <literal>CS_CLIENT</literal> or <literal>CS_SERVER</literal>.
- </para>
-
- <synopsis>
- int cs_listen(COMSTACK handle, char *addr, int *addrlen);
- </synopsis>
-
- <para>
- Call this to process incoming events on an endpoint that has been
- bound in listening mode. It will return 0 to indicate that the connect
- request has been received, 1 to signal a partial reception, and -1 to
- indicate an error condition.
- </para>
-
- <synopsis>
- COMSTACK cs_accept(COMSTACK handle);
- </synopsis>
-
- <para>
- This finalizes the server-side association establishment, after
- cs_listen has completed successfully. It returns a new connection
- endpoint, which represents the new association. The application will
- typically wish to fork off a process to handle the association at this
- point, and continue listen for new connections on the old
- <literal>handle</literal>.
- </para>
-
- <para>
- You can use the call
- </para>
-
- <synopsis>
- const char *cs_addrstr(COMSTACK);
- </synopsis>
-
- <para>
- on an established connection to retrieve the host-name of the remote host.
- </para>
-
- <note>
- <para>You may need to use this function with some care if your
- name server service is slow or unreliable
- </para>
- </note>
-
- </sect1>
- <sect1 id="comstack.addresses"><title>Addresses</title>
-
- <para>
- The low-level format of the addresses are different depending on the
- mode of communication you have chosen. A function is provided by each
- of the lower layers to map a user-friendly string-form address to the
- binary form required by the lower layers.
- </para>
-
- <synopsis>
- void *cs_straddr(COMSTACK handle, const char *str);
- </synopsis>
-
- <para>
- The format for TCP/IP and SSL addresses is:
- </para>
-
- <synopsis>
- <host> [ ':' <portnum> ]
- </synopsis>
-
- <para>
- The <literal>hostname</literal> can be either a domain name or an
- IP address. The port number, if omitted, defaults to 210.
- </para>
-
- <para>
- For TCP/IP and SSL, the special hostnames <literal>@</literal>,
- maps to <literal>IN6ADDR_ANY_INIT</literal> with
- IPV4 binding as well (bindv6only=0),
- The special hostname <literal>@4</literal> binds to
- <literal>INADDR_ANY</literal> (IPV4 only listener).
- The special hostname <literal>@6</literal> binds to
- <literal>IN6ADDR_ANY_INIT</literal> with bindv6only=1 (IPV6 only listener).
- </para>
-
- <para>
- For UNIX sockets, the format of an address is the socket filename.
- </para>
-
- <para>
- When a connection has been established, you can use
- </para>
-
- <synopsis>
- const char *cs_addrstr(COMSTACK h);
- </synopsis>
-
- <para>
- to retrieve the host name of the peer system. The function returns
- a pointer to a static area, which is overwritten on the next call
- to the function.
- </para>
-
- <para>
- A fairly recent addition to the &comstack; module is the utility
- function
- </para>
- <synopsis>
- COMSTACK cs_create_host (const char *str, int blocking, void **vp);
- </synopsis>
- <para>
- which is just a wrapper for <function>cs_create</function> and
- <function>cs_straddr</function>. The <parameter>str</parameter>
- is similar to that described for <function>cs_straddr</function>
- but with a prefix denoting the &comstack; type. Prefixes supported
- are <literal>tcp:</literal>, <literal>unix:</literal> and
- <literal>ssl:</literal> for TCP/IP, UNIX and SSL respectively.
- If no prefix is given, then TCP/IP is used.
- The <parameter>blocking</parameter> is passed to
- function <function>cs_create</function>. The third parameter
- <parameter>vp</parameter> is a pointer to &comstack; stack type
- specific values.
- Parameter <parameter>vp</parameter> is reserved for future use.
- Set it to <literal>NULL</literal>.
- </para>
-
- </sect1>
-
- <sect1 id="comstack.ssl"><title>SSL</title>
- <para>
- <synopsis>
- void *cs_get_ssl(COMSTACK cs);
- </synopsis>
- Returns the SSL handle, <literal>SSL *</literal> for comstack. If comstack
- is not of type SSL, NULL is returned.
- </para>
-
- <para>
- <synopsis>
- int cs_set_ssl_ctx(COMSTACK cs, void *ctx);
- </synopsis>
- Sets SSL context for comstack. The parameter is expected to be of type
- <literal>SSL_CTX *</literal>. This function should be called just
- after comstack has been created (before connect, bind, etc).
- This function returns 1 for success; 0 for failure.
- </para>
-
- <para>
- <synopsis>
- int cs_set_ssl_certificate_file(COMSTACK cs, const char *fname);
- </synopsis>
- Sets SSL certificate for comstack as a PEM file. This function
- returns 1 for success; 0 for failure.
- </para>
-
-
- <para>
- <synopsis>
- int cs_get_ssl_peer_certificate_x509(COMSTACK cs, char **buf, int *len);
- </synopsis>
- This function returns the peer certificate. If successful,
- <literal>*buf</literal> and <literal>*len</literal> holds
- X509 buffer and length respectively. Buffer should be freed
- with <literal>xfree</literal>. This function returns 1 for success;
- 0 for failure.
- </para>
-
- </sect1>
-
- <sect1 id="comstack.diagnostics"><title>Diagnostics</title>
-
- <para>
- All functions return -1 if an error occurs. Typically, the functions
- will return 0 on success, but the data exchange functions
- (<function>cs_get</function>, <function>cs_put</function>,
- <function>cs_more</function>) follow special rules. Consult their
- descriptions.
- </para>
-
- <para>
- The error code for the COMSTACK can be retrieved using C macro
- <function>cs_errno</function> which will return one
- of the error codes <literal>CSYSERR</literal>,
- <literal>CSOUTSTATE</literal>,
- <literal>CSNODATA</literal>, ...
- </para>
-
- <synopsis>
- int cs_errno(COMSTACK handle);
- </synopsis>
-
- <para>
- You can the textual representation of the error code
- by using <function>cs_errmsg</function> - which
- works like <function>strerror(3)</function>
- </para>
-
- <synopsis>
- const char *cs_errmsg(int n);
- </synopsis>
-
- <para>
- It is also possible to get straight to the textual represenataion
- without the error code by using
- <function>cs_strerror</function>.
- </para>
-
- <synopsis>
- const char *cs_strerror(COMSTACK h);
- </synopsis>
-
- </sect1>
- <sect1 id="comstack.summary"><title>Summary and Synopsis</title>
-
- <synopsis><![CDATA[
- #include <yaz/comstack.h>
-
- #include <yaz/tcpip.h> /* this is for TCP/IP and SSL support */
- #include <yaz/unix.h> /* this is for UNIX socket support */
-
- COMSTACK cs_create(CS_TYPE type, int blocking, int protocol);
-
- COMSTACK cs_createbysocket(int s, CS_TYPE type, int blocking,
- int protocol);
- COMSTACK cs_create_host(const char *str, int blocking,
- void **vp);
-
- int cs_bind(COMSTACK handle, int mode);
-
- int cs_connect(COMSTACK handle, void *address);
-
- int cs_rcvconnect(COMSTACK handle);
-
- int cs_listen(COMSTACK handle);
-
- COMSTACK cs_accept(COMSTACK handle);
-
- int cs_put(COMSTACK handle, char *buf, int len);
-
- int cs_get(COMSTACK handle, char **buf, int *size);
-
- int cs_more(COMSTACK handle);
-
- void cs_close(COMSTACK handle);
-
- int cs_look(COMSTACK handle);
-
- void *cs_straddr(COMSTACK handle, const char *str);
-
- const char *cs_addrstr(COMSTACK h);
-]]>
- </synopsis>
- </sect1>
-
- </chapter>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
-
+++ /dev/null
- <appendix id="credits"><title>Credits</title>
- <para>
- This appendix lists individuals that have contributed in the development
- of &yaz;. Some have contributed with code, while others have provided bug
- fixes or suggestions. If we're missing somebody, of if you, for
- whatever reason, don't like to be listed here, let us know.
- </para>
-
- <itemizedlist>
- <listitem><para>
- Gary Anderson
- </para></listitem>
- <listitem><para>
- Dimitrios Andreadis
- </para></listitem>
- <listitem><para>
- Morten Bøgeskov
- </para></listitem>
- <listitem><para>
- Rocco Carbone
- </para></listitem>
- <listitem><para>
- Matthew Carey
- </para></listitem>
- <listitem><para>
- Hans van Dalen
- </para></listitem>
- <listitem><para>
- Irina Dijour
- </para></listitem>
- <listitem><para>
- Larry E. Dixson
- </para></listitem>
- <listitem><para>
- Hans van den Dool
- </para></listitem>
- <listitem><para>
- Mads Bondo Dydensborg
- </para></listitem>
- <listitem><para>
- Franck Falcoz
- </para></listitem>
- <listitem><para>
- Kevin Gamiel
- </para></listitem>
- <listitem><para>
- Morten Garkier Hendriksen
- </para></listitem>
- <listitem><para>
- Morten Holmqvist
- </para></listitem>
- <listitem><para>
- Ian Ibbotson
- </para></listitem>
- <listitem><para>
- Shigeru Ishida
- </para></listitem>
- <listitem><para>
- Heiko Jansen
- </para></listitem>
- <listitem><para>
- David Johnson
- </para></listitem>
- <listitem><para>
- Oleg Kolobov
- </para></listitem>
- <listitem><para>
- Giannis Kosmas
- </para></listitem>
- <listitem><para>
- Kang-Jin Lee
- </para></listitem>
- <listitem><para>
- Pieter Van Lierop
- </para></listitem>
- <listitem><para>
- Stefan Lohrum
- </para></listitem>
- <listitem><para>
- Ronald van der Meer
- </para></listitem>
- <listitem><para>
- Thomas W. Place
- </para></listitem>
- <listitem><para>
- Peter Popovics
- </para></listitem>
- <listitem><para>
- Jacob Chr. Poulsen
- </para></listitem>
- <listitem><para>
- Ko van der Sloot
- </para></listitem>
- <listitem><para>
- Mike Taylor
- </para></listitem>
- <listitem><para>
- Rustam T. Usmanov
- </para></listitem>
- <listitem><para>
- Charles Woodfield
- </para></listitem>
- <listitem><para>
- Tom André Øverland
- </para></listitem>
- </itemizedlist>
- </appendix>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
<!ENTITY yaz "YAZ">
-<!ENTITY bookinfo SYSTEM "bookinfo.xml">
-<!ENTITY chap-introduction SYSTEM "introduction.xml">
-<!ENTITY chap-installation SYSTEM "installation.xml">
-<!ENTITY chap-zoom SYSTEM "zoom.xml">
-<!ENTITY chap-asn SYSTEM "asn.xml">
-<!ENTITY chap-soap SYSTEM "soap.xml">
-<!ENTITY chap-tools SYSTEM "tools.xml">
-<!ENTITY chap-odr SYSTEM "odr.xml">
-<!ENTITY chap-comstack SYSTEM "comstack.xml">
-<!ENTITY chap-server SYSTEM "server.xml">
-<!ENTITY chap-future SYSTEM "future.xml">
<!ENTITY std-oid-table SYSTEM "std-oid-table.xml">
<!ENTITY bib1-diag-table SYSTEM "bib1-diag-table.xml">
<!ENTITY srw-diag-table SYSTEM "srw-diag-table.xml">
-<!ENTITY app-license SYSTEM "license.xml">
-<!ENTITY app-indexdata SYSTEM "indexdata.xml">
-<!ENTITY app-credits SYSTEM "credits.xml">
<!ENTITY gfs-options SYSTEM "gfs-options.xml">
<!ENTITY gfs-synopsis SYSTEM "gfs-synopsis.xml">
<!ENTITY gfs-virtual SYSTEM "gfs-virtual.xml">
+++ /dev/null
- <chapter id="future"><title>Future Directions</title>
-
- <para>
- We have a new and better version of the front-end server on the drawing
- board. Resources and external commitments will govern when we'll be
- able to do something real with it. Features should include greater
- flexibility, greater support for access/resource control, and easy
- support for Explain (possibly with Zebra as an extra database engine).
- </para>
-
- <para>
- &yaz; is a BER toolkit and as such should support all protocols
- out there based on that. We'd like to see running ILL applications.
- It shouldn't be that hard. Another thing that would be interesting is
- LDAP. Maybe a generic framework for doing IR using both LDAP and
- Z39.50 transparently.
- </para>
-
- <para>
- The SOAP implementation is incomplete. In the future we hope
- to add more features to it. Perhaps make a WSDL/XML Schema compiler.
- The authors of libxml2 are already working on XML Schema / RelaxNG
- compilers so this may not be too hard.
- </para>
-
- <para>
- It would be neat to have a proper module mechanism for the Generic
- Frontend Server so that backend would be dynamically
- loaded (as shared objects / DLLs).
- </para>
-
- <para>
- Other than that, &yaz; generally moves in the directions which appear to
- make the most people happy (including ourselves, as prime users of the
- software). If there's something you'd like to see in here, then drop
- us a note and let's see what we can come up with.
- </para>
-
- </chapter>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
-->
<variablelist>
-
- <varlistentry><term><literal>-a </literal>
- <replaceable>file</replaceable></term>
+ <varlistentry>
+ <term><literal>-a </literal>
+ <replaceable>file</replaceable></term>
<listitem><para>
- Specify a file for dumping PDUs (for diagnostic purposes).
+ Specify a file for dumping PDUs (for diagnostic purposes).
The special name <literal>-</literal> (dash) sends output to
<literal>stderr</literal>.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-S</literal></term>
- <listitem><para>
- Don't fork or make threads on connection requests. This is good for
- debugging, but not recommended for real operation: Although the
- server is asynchronous and non-blocking, it can be nice to keep
- a software malfunction (okay then, a crash) from affecting all
- current users.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-1</literal></term>
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-S</literal></term>
+ <listitem><para>
+ Don't fork or make threads on connection requests. This is good for
+ debugging, but not recommended for real operation: Although the
+ server is asynchronous and non-blocking, it can be nice to keep
+ a software malfunction (okay then, a crash) from affecting all
+ current users.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-1</literal></term>
<listitem><para>
- Like <literal>-S</literal> but after one session the server
- exits. This mode is for debugging <emphasis>only</emphasis>.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-T</literal></term>
+ Like <literal>-S</literal> but after one session the server
+ exits. This mode is for debugging <emphasis>only</emphasis>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-T</literal></term>
<listitem><para>
- Operate the server in threaded mode. The server creates a thread
- for each connection rather than a fork a process. Only available
- on UNIX systems that offers POSIX threads.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-s</literal></term>
+ Operate the server in threaded mode. The server creates a thread
+ for each connection rather than a fork a process. Only available
+ on UNIX systems that offers POSIX threads.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-s</literal></term>
<listitem><para>
- Use the SR protocol (obsolete).
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-z</literal></term>
- <listitem><para>
- Use the Z39.50 protocol (default). This option and <literal>-s</literal>
- complement each other.
- You can use both multiple times on the same command
- line, between listener-specifications (see below). This way, you
- can set up the server to listen for connections in both protocols
- concurrently, on different local ports.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-l </literal>
- <replaceable>file</replaceable></term>
+ Use the SR protocol (obsolete).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-z</literal></term>
+ <listitem><para>
+ Use the Z39.50 protocol (default). This option and <literal>-s</literal>
+ complement each other.
+ You can use both multiple times on the same command
+ line, between listener-specifications (see below). This way, you
+ can set up the server to listen for connections in both protocols
+ concurrently, on different local ports.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-l </literal><replaceable>file</replaceable></term>
<listitem><para>The logfile.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-c </literal>
- <replaceable>config</replaceable></term>
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-c </literal><replaceable>config</replaceable></term>
<listitem><para>A user option that serves as a specifier for some
- sort of configuration, usually a filename.
- The argument to this option is transferred to member
- <literal>configname</literal> of the
- <literal>statserv_options_block</literal>.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-f </literal>
- <replaceable>vconfig</replaceable></term>
+ sort of configuration, usually a filename.
+ The argument to this option is transferred to member
+ <literal>configname</literal> of the
+ <literal>statserv_options_block</literal>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-f </literal><replaceable>vconfig</replaceable></term>
<listitem><para>This specifies an XML file that describes
- one or more YAZ frontend virtual servers.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-C </literal>
- <replaceable>fname</replaceable></term>
+ one or more YAZ frontend virtual servers.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-C </literal><replaceable>fname</replaceable></term>
<listitem><para>Sets SSL certificate file name for server (PEM).
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-v </literal>
- <replaceable>level</replaceable></term>
- <listitem><para>
- The log level. Use a comma-separated list of members of the set
- {fatal,debug,warn,log,malloc,all,none}.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-u </literal>
- <replaceable>uid</replaceable></term>
- <listitem><para>
- Set user ID. Sets the real UID of the server process to that of the
- given user. It's useful if you aren't comfortable with having the
- server run as root, but you need to start it as such to bind a
- privileged port.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-w </literal>
- <replaceable>dir</replaceable></term>
- <listitem><para>
- The server changes to this directory during before listening
- on incoming connections. This option is useful
- when the server is operating from the <application>inetd</application>
- daemon (see <literal>-i</literal>).
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-p </literal>
- <replaceable>pidfile</replaceable></term>
- <listitem><para>
- Specifies that the server should write its Process ID to
- file given by <replaceable>pidfile</replaceable>.
- A typical location would be <filename>/var/run/yaz-ztest.pid</filename>.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-i</literal></term>
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-v </literal><replaceable>level</replaceable></term>
<listitem><para>
- Use this to make the the server run from the
- <application>inetd</application> server (UNIX only).
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-D</literal></term>
+ The log level. Use a comma-separated list of members of the set
+ {fatal,debug,warn,log,malloc,all,none}.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-u </literal><replaceable>uid</replaceable></term>
+ <listitem><para>
+ Set user ID. Sets the real UID of the server process to that of the
+ given user. It's useful if you aren't comfortable with having the
+ server run as root, but you need to start it as such to bind a
+ privileged port.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-w </literal><replaceable>dir</replaceable></term>
+ <listitem><para>
+ The server changes to this directory during before listening
+ on incoming connections. This option is useful
+ when the server is operating from the <application>inetd</application>
+ daemon (see <literal>-i</literal>).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry><term>
+ <literal>-p </literal><replaceable>pidfile</replaceable>
+ </term>
+ <listitem><para>
+ Specifies that the server should write its Process ID to
+ file given by <replaceable>pidfile</replaceable>.
+ A typical location would be <filename>/var/run/yaz-ztest.pid</filename>.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-i</literal></term>
<listitem><para>
- Use this to make the server put itself in the background and
- run as a daemon. If neither <literal>-i</literal> nor
- <literal>-D</literal> is given, the server starts in the foreground.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-install</literal></term>
+ Use this to make the the server run from the
+ <application>inetd</application> server (UNIX only).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-D</literal></term>
<listitem><para>
- Use this to install the server as an NT service
- (Windows NT/2000/XP only).
- Control the server by going to the Services in the Control Panel.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-installa</literal></term>
+ Use this to make the server put itself in the background and
+ run as a daemon. If neither <literal>-i</literal> nor
+ <literal>-D</literal> is given, the server starts in the foreground.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-install</literal></term>
<listitem><para>
- Use this to install and activate the server as an NT service
- (Windows NT/2000/XP only).
- Control the server by going to the Services in the Control Panel.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-remove</literal></term>
+ Use this to install the server as an NT service
+ (Windows NT/2000/XP only).
+ Control the server by going to the Services in the Control Panel.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-installa</literal></term>
<listitem><para>
- Use this to remove the server from the NT services
- (Windows NT/2000/XP only).
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-t </literal>
- <replaceable>minutes</replaceable></term>
+ Use this to install and activate the server as an NT service
+ (Windows NT/2000/XP only).
+ Control the server by going to the Services in the Control Panel.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-remove</literal></term>
<listitem><para>
- Idle session timeout, in minutes.
- </para></listitem></varlistentry>
-
- <varlistentry><term><literal>-k </literal>
- <replaceable>size</replaceable></term>
+ Use this to remove the server from the NT services
+ (Windows NT/2000/XP only).
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-t </literal><replaceable>minutes</replaceable></term>
<listitem><para>
- Maximum record size/message size, in kilobytes.
- </para></listitem>
+ Idle session timeout, in minutes.
+ </para></listitem>
</varlistentry>
-
- <varlistentry><term><literal>-K</literal></term>
+ <varlistentry>
+ <term><literal>-k </literal><replaceable>size</replaceable></term>
<listitem><para>
- Forces no-keepalive for HTTP sessions. By default GFS will keep
- sessions alive for HTTP 1.1 sessions (as defined by the standard).
- Using this option will force GFS to close the connection for each
- operation.
- </para></listitem>
+ Maximum record size/message size, in kilobytes.
+ </para></listitem>
</varlistentry>
-
- <varlistentry><term><literal>-r </literal>
- <replaceable>size</replaceable></term>
+ <varlistentry>
+ <term><literal>-K</literal></term>
+ <listitem><para>
+ Forces no-keepalive for HTTP sessions. By default GFS will keep
+ sessions alive for HTTP 1.1 sessions (as defined by the standard).
+ Using this option will force GFS to close the connection for each
+ operation.
+ </para></listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>-r </literal><replaceable>size</replaceable></term>
<listitem><para>
- Maximum size of log file before rotation occurs, in kilobytes.
- Default size is 1048576 k (=1 GB).
- </para></listitem>
+ Maximum size of log file before rotation occurs, in kilobytes.
+ Default size is 1048576 k (=1 GB).
+ </para></listitem>
</varlistentry>
-
- <varlistentry><term><literal>-d </literal>
- <replaceable>daemon</replaceable></term>
- <listitem><para>
- Set name of daemon to be used in hosts access file.
- See
- <citerefentry>
- <refentrytitle>hosts_access</refentrytitle>
- <manvolnum>5</manvolnum>
- </citerefentry>
- and
- <citerefentry>
- <refentrytitle>tcpd</refentrytitle>
- <manvolnum>8</manvolnum>
+ <varlistentry>
+ <term>
+ <literal>-d </literal><replaceable>daemon</replaceable>
+ </term>
+ <listitem><para>
+ Set name of daemon to be used in hosts access file.
+ See
+ <citerefentry>
+ <refentrytitle>hosts_access</refentrytitle>
+ <manvolnum>5</manvolnum>
+ </citerefentry>
+ and
+ <citerefentry>
+ <refentrytitle>tcpd</refentrytitle>
+ <manvolnum>8</manvolnum>
</citerefentry>.
- </para></listitem>
+ </para></listitem>
</varlistentry>
-
- <varlistentry><term><literal>-m </literal>
- <replaceable>time-format</replaceable></term>
+ <varlistentry>
+ <term><literal>-m </literal><replaceable>time-format</replaceable></term>
<listitem><para>
Sets the format of time-stamps in the log-file. Specify a string in
the input format to <literal>strftime()</literal>.
- </para></listitem>
+ </para></listitem>
</varlistentry>
-
- <varlistentry><term><literal>-V </literal></term>
+ <varlistentry>
+ <term><literal>-V </literal></term>
<listitem><para>
Display YAZ version and exit.
- </para></listitem>
+ </para></listitem>
</varlistentry>
-
</variablelist>
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document: "yaz.xml"
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<arg choice="opt"><option>-ziDSTV1</option></arg>
<arg choice="opt" rep="repeat">listener-spec</arg>
</cmdsynopsis>
-
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document: "yaz.xml"
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document: "yaz.xml"
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
+++ /dev/null
- <appendix id="indexdata"><title>About Index Data</title>
-
- <para>
- Index Data is a consulting and software-development enterprise that
- specializes in library and information management systems. Our
- interests and expertise span a broad range of related fields, and one
- of our primary, long-term objectives is the development of a powerful
- information management
- system with open network interfaces and hyper-media capabilities.
- </para><para>
- We make this software available free of charge, on a fairly unrestrictive
- license; as a service to the networking community, and to further the
- development of quality software for open network communication.
- </para><para>
- We'll be happy to answer questions about the software, and about ourselves
- in general.
- </para>
- <para>
- <address>
- Index Data ApS
- <street>Købmagergade 43 2.</street>
- <postcode>1150 Copenhagen K</postcode>
- <country>Denmark</country>
- Phone <phone>+45 3341 0100</phone>
- Fax <fax>+45 3341 0101</fax>
- Email <email>info@indexdata.dk</email>
- </address>
-
- </para>
- <para>
- The Hacker's Jargon File has the following to say about the
- use of the
- prefix "YA" in the name of a software product.
- </para>
- <para>
- <citation>
- Yet Another. adj. 1. Of your own work: A
- humorous allusion often used in titles to acknowledge that the
- topic is not original, though the content is. As in "Yet Another
- AI Group" or "Yet Another Simulated Annealing Algorithm".
- 2. Of
- others' work: Describes something of which there are already far
- too many.
- </citation>
-
- </para>
- </appendix>
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
+++ /dev/null
-<chapter id="installation"><title>Compilation and Installation</title>
-
- <sect1 id="installation-introduction"><title>Introduction</title>
- <para>
- The latest version of the software will generally be found at:
- </para>
- <para>
- <ulink url="&url.yaz.download;"/>
- </para>
- <para>
- We have tried our best to keep the software portable, and on many
- platforms, you should be able to compile everything with little or
- no changes.
- </para>
-
- <para>
- The software is regularly tested on
- <ulink url="&url.debian;">Debian GNU/Linux</ulink>,
- <ulink url="&url.centos;">CentOS</ulink>,
- <ulink url="&url.ubuntu;">Ubuntu Linux</ulink>,
- <ulink url="&url.freebsd;">FreeBSD (i386)</ulink>,
- <ulink url="&url.macosx;">MAC OSX</ulink>,
- <ulink url="&url.solaris;">Solaris</ulink>,
- Windows 7, Windows XP.
- </para>
-
- <para>
- Some versions have be known to work on HP/UX,
- DEC Unix, <ulink url="&url.netbsd;">NetBSD</ulink>,
- <ulink url="&url.openbsd;">OpenBSD</ulink>,
- IBM AIX,
- Data General DG/UX (with some CFLAGS tinkering),
- SGI/IRIX, DDE Supermax, Apple Macintosh (using the Codewarrior programming
- environment and the GUSI socket libraries),
- IBM AS/400 .
- </para>
-
- <para>
- If you move the software to other platforms, we'd be grateful if you'd
- let us know about it. If you run into difficulties, we will try to help
- if we can, and if you solve the problems, we would be happy to include
- your fixes in the next release. So far, we have mostly avoided
- <literal>#ifdefs</literal> for individual platforms, and we'd
- like to keep it that way as far as it makes sense.
- </para>
-
- <para>
- We maintain a mailing-list for the purpose of announcing new releases and
- bug-fixes, as well as general discussion. Subscribe by
- filling-in the form
- <ulink url="&url.yaz.mailinglist;">here</ulink>.
- General questions and problems can be directed at
- <ulink url="&url.yaz.mail;"/>, or the address given at the top of
- this document.
- </para>
-
- </sect1>
- <sect1 id="installation.unix"><title>UNIX</title>
-
- <para>
- We provide
- <ulink url="&url.debian;">Debian GNU/Linux</ulink> (i386 and amd64),
- <ulink url="&url.ubuntu;">Ubuntu</ulink> (i386 and amd64)
- and
- <ulink url="&url.centos;">CentOS</ulink> (amd64 only) packages for &yaz;.
- You should be able to create packages for other CPUs by building
- them from the source package.
- </para>
-
- <para>
- YAZ is also part of several packages repositories. Some of them are
- </para>
-
- <itemizedlist>
- <listitem>
- <para>
- Solaris CSW: <ulink url="http://www.opencsw.org/packages/yaz/"/>
- </para>
- </listitem>
- <listitem>
- <para>
- Solaris: <ulink url="http://unixpackages.com"/>
- </para>
- </listitem>
- <listitem>
- <para>
- FreeBSD: <ulink url="http://www.freshports.org/net/yaz"/>
- </para>
- </listitem>
- <listitem>
- <para>
- Debian: <ulink url="http://packages.debian.org/search?keywords=yaz"/>
- </para>
- </listitem>
- <listitem>
- <para>
- Ubuntu: <ulink url="https://launchpad.net/ubuntu/+source/yaz"/>
- </para>
- </listitem>
- <listitem>
- <para>
- NetBSD:
- <ulink url="http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/net/yaz/README.html"/>
- </para>
- </listitem>
- </itemizedlist>
-
- <sect2 id="installation.source.unix">
- <title>Compiling from source on Unix</title>
-
- <para>
- Note that if your system doesn't have a native ANSI C compiler, you may
- have to acquire one separately. We recommend
- <ulink url="&url.gcc;">GCC</ulink>.
- </para>
-
- <para>
- If you wish to use character set conversion facilities in &yaz; or if you
- are compiling &yaz; for use with Zebra it is a good idea to ensure that
- the iconv library is installed. Some Unixes today already have it
- - if not, we suggest
- <ulink url="&url.libiconv;">GNU libiconv</ulink>.
- </para>
-
- <para>
- YAZ 3.0.16 and later includes a wrapper for the
- <ulink url="&url.icu;">ICU</ulink>
- (International Components for Unicode).
- In order to use this, the developer version of the ICU library
- must be available. ICU support is recommended for applications
- such as Pazpar2 and Zebra.
- </para>
-
- <para>
- The <ulink url="&url.libxslt;">libxslt</ulink>,
- <ulink url="&url.libxml2;">libxml2</ulink> librararies are required
- if &yaz; is to support SRU/Solr.
- These libraries are very portable and should compile out-of-the
- box on virtually all Unix platforms. It is available in binary
- forms for Linux and others.
- </para>
-
- <para>
- The GNU tools
- <ulink url="&url.autoconf;">Autoconf</ulink>,
- <ulink url="&url.automake;">Automake</ulink> and
- <ulink url="&url.libtool;">Libtool</ulink>
- are used to generate Makefiles and configure &yaz; for the system.
- You do <emphasis>not</emphasis> these tools unless you're using the
- Git version of &yaz;.
- </para>
-
- <para>
- The CQL parser for &yaz; is built using
- GNU <ulink url="&url.bison;">Bison</ulink>.
- This tool is only needed if you're using the Git version of &yaz;.
- </para>
-
- <para>
- &yaz; includes a tiny ASN.1 compiler. This compiler is
- written in <ulink url="&url.tcl;">Tcl</ulink>.
- But as for Bison you do not need it unless you're using Git
- version of &yaz; or you're using the compiler to built own codecs
- for private ASN.1.
- </para>
-
- <para>
- Generally it should be sufficient to run configure without options,
- like this:
- </para>
-
- <screen>
- ./configure
- </screen>
-
- <para>
- The configure script attempts to use use the C compiler specified by
- the <literal>CC</literal> environment variable. If not set, GNU C will be
- used if it is available. The <literal>CFLAGS</literal> environment
- variable holds options to be passed to the C compiler. If you're using
- Bourne-compatible shell you may pass something like this to use a
- particular C compiler with optimization enabled:
- </para>
-
- <screen>
- CC=/opt/ccs/bin/cc CFLAGS=-O ./configure
- </screen>
-
- <para>
- To customize &yaz;, the configure script also accepts a set of options.
- The most important are:
-
- <variablelist>
- <varlistentry><term>
- <literal>--prefix</literal>=<replaceable>prefix</replaceable></term>
- <listitem><para>Specifies installation prefix for &yaz;. This is
- only needed if you run <literal>make install</literal> later to
- perform a "system" installation. The prefix is
- <literal>/usr/local</literal> if not specified.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>--enable-tcpd</literal></term>
- <listitem><para>The front end server will be built using Wietse's
- <ulink url="&url.tcpwrapper;">TCP wrapper library</ulink>.
- It allows you to allow/deny clients depending on IP number.
- The TCP wrapper library is often used in GNU/Linux and
- BSD distributions.
- See
- <citerefentry>
- <refentrytitle>hosts_access</refentrytitle>
- <manvolnum>5</manvolnum>
- </citerefentry>
- and
- <citerefentry>
- <refentrytitle>tcpd</refentrytitle>
- <manvolnum>8</manvolnum>
- </citerefentry>.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><literal>--enable-threads</literal></term>
- <listitem><para>&yaz; will be built using POSIX threads.
- Specifically, <constant>_REENTRANT</constant> will be defined during
- compilation.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><literal>--disable-shared</literal></term>
- <listitem><para>The make process will not create shared
- libraries (also known as shared objects <filename>.so</filename>).
- By default, shared libraries are created -
- equivalent to <literal>--enable-shared</literal>.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><literal>--disable-shared</literal></term>
- <listitem><para>The make process will not create
- static libraries (<filename>.a</filename>).
- By default, static libraries are created -
- equivalent to <literal>--enable-static</literal>.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>--with-iconv</literal>[=<replaceable>prefix</replaceable>]
- </term>
- <listitem><para>Compile &yaz; with iconv library in directory
- <replaceable>prefix</replaceable>. By default configure will
- search for iconv on the system. Use this option if it
- doesn't find iconv. Alternatively,
- <literal>--without-iconv</literal>, can be uset to force &yaz;
- not to use iconv.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>--with-xslt</literal>[=<replaceable>prefix</replaceable>]
- </term>
- <listitem><para>Compile &yaz; with
- <ulink url="&url.libxslt;">libxslt</ulink> in directory
- <replaceable>prefix</replaceable>.
- Use this option if you want XSLT and XML support.
- By default, configure will
- search for libxslt on the system. Use this option if it
- libxslt is not found automatically. Alternatively,
- <literal>--without-xslt</literal>, can be used to force &yaz;
- not to use libxslt.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>--with-xml2</literal>[=<replaceable>prefix</replaceable>]
- </term>
- <listitem><para>Compile &yaz; with
- <ulink url="&url.libxml2;">libxml2</ulink> in directory
- <replaceable>prefix</replaceable>.
- Use this option if you want &yaz; to use XML and support SRU/Solr.
- By default, configure will
- search for libxml2 on the system. Use this option if it
- libxml2 is not found automatically. Alternatively,
- <literal>--without-xml2</literal>, can be used to force &yaz;
- not to use libxml2.
- </para>
- <para>
- Note that option <literal>--with-xslt</literal>
- also enables libxml2.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>--with-gnutls</literal>[=<replaceable>prefix</replaceable>]
- </term>
- <listitem><para>&yaz; will be linked with the GNU TLS libraries and
- an SSL COMSTACK will be provided. By default configure enables
- SSL support for YAZ if the GNU TLS development libraries are found
- on the system.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>--with-icu</literal>[=<replaceable>prefix</replaceable>]
- </term>
- <listitem><para>&yaz; will be linked the
- <ulink url="&url.icu;">ICU</ulink> library in the prefix if given.
- If prefix is not given, the libraries exposed by the script
- <application>icu-config</application> will be used if found.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>--with-libgcrypt</literal>[=<replaceable>prefix</replaceable>]
- </term>
- <listitem><para>&yaz; will be linked with
- <ulink url="&url.libgcrypt;">Libgcrypt</ulink> in the prefix if given.
- If prefix is not given, the libraries exposed by the script
- <application>libgcrypt-config</application> will be used if found.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>--with-memcached</literal>
- </term>
- <listitem><para>&yaz; will be linked with
- <ulink url="&url.libmemcached;">libMemcached</ulink> to allow
- for result-set caching for ZOOM.
- The prefix can not be given. Note that YAZ will only search
- for libMemcached if Libgcrypt is also enabled.
- </para></listitem>
- </varlistentry>
-
- </variablelist>
-
- </para>
- <para>
- When configured, build the software by typing:
- <screen>
- make
- </screen>
- </para>
-
- <para>
- The following files are generated by the make process:
- <variablelist>
- <varlistentry><term><filename>src/libyaz.la</filename></term>
- <listitem><para>
- Main &yaz; library. This is no ordinary library. It's
- a Libtool archive.
- By default, &yaz; creates a static library in
- <filename>lib/.libs/libyaz.a</filename>.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>src/libyaz_server.la</filename></term>
- <listitem><para>
- Generic Frontend server. This is an add-on for libyaz.la.
- Code in this library uses POSIX threads functions - if POSIX
- threads are available on the platform.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>src/libyaz_icu.la</filename></term>
- <listitem><para>
- Functions that wrap the ICU library.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>ztest/yaz-ztest</filename></term>
- <listitem><para>Test Z39.50 server.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>client/yaz-client</filename></term>
- <listitem><para>Z39.50 client for testing the protocol.
- See chapter <link linkend="yaz-client">
- YAZ client</link> for more information.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>util/yaz-config</filename></term>
- <listitem><para>A Bourne-shell script, generated by configure, that
- specifies how external applications should compile - and link with
- &yaz;.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>util/yaz-asncomp</filename></term>
- <listitem><para>The ASN.1 compiler for &yaz;. Requires the
- Tcl Shell, <application>tclsh</application>, in
- <literal>PATH</literal> to operate.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>util/yaz-iconv</filename></term>
- <listitem><para>This program converts data in one character set to
- another. This command exercises the YAZ character set
- conversion API.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>util/yaz-marcdump</filename></term>
- <listitem><para>This program parses ISO2709 encoded MARC records
- and prints them in line-format or XML.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>util/yaz-icu</filename></term>
- <listitem><para>This program exposes the ICU wrapper library if that
- is enabled for YAZ. Only if ICU is available this program is
- useful.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>util/yaz-url</filename></term>
- <listitem><para>This program is a simple HTTP page fetcher ala
- wget or curl.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>zoom/zoomsh</filename></term>
- <listitem><para>
- A simple shell implemented on top of the
- <link linkend="zoom">ZOOM</link> functions.
- The shell is a command line application that allows you to enter
- simple commands to perform ZOOM operations.
- </para></listitem>
- </varlistentry>
-
- <varlistentry><term><filename>zoom/zoomtst1</filename>,
- <filename>zoom/zoomtst2</filename>, ..</term>
- <listitem><para>
- Several small applications that demonstrates the ZOOM API.
- </para></listitem>
- </varlistentry>
- </variablelist>
-
- </para>
-
- <para>
- If you wish to install &yaz; in system directories
- <filename>/usr/local/bin</filename>,
- <filename>/usr/local/lib</filename> .. etc, you can type:
- </para>
-
- <screen>
- make install
- </screen>
-
- <para>
- You probably need to have root access in order to perform this.
- You must specify the <literal>--prefix</literal> option for configure if
- you wish to install &yaz; in other directories than the default
- <filename>/usr/local/</filename>.
- </para>
-
- <para>
- If you wish to perform an un-installation of &yaz;, use:
- </para>
-
- <screen>
- make uninstall
- </screen>
-
- <para>
- This will only work if you haven't reconfigured &yaz; (and therefore
- changed installation prefix). Note that uninstall will not
- remove directories created by make install, e.g.
- <filename>/usr/local/include/yaz</filename>.
- </para>
- </sect2>
-
- <sect2 id="installation-linking-yaz-unix">
- <title>How to make apps using YAZ on UNIX</title>
- <para>
- This section describes how to compile - and link your own
- applications using the &yaz; toolkit.
- If you're used to Makefiles this shouldn't be hard. As for
- other libraries you have used before, you have to set a proper include
- path for your C/C++ compiler and specify the location of
- &yaz; libraries. You can do it by hand, but generally we suggest
- you use the <filename>yaz-config</filename> that is generated
- by <filename>configure</filename>. This is especially
- important if you're using the threaded version of &yaz; which
- require you to pass more options to your linker/compiler.
- </para>
- <para>
- The <filename>yaz-config</filename> script accepts command line
- options that makes the <filename>yaz-config</filename> script print
- options that you should use in your make process.
- The most important ones are:
- <literal>--cflags</literal>, <literal>--libs</literal>
- which prints C compiler flags, and linker flags respectively.
- </para>
- <para>
- A small and complete <literal>Makefile</literal> for a C
- application consisting of one source file,
- <filename>myprog.c</filename>, may look like this:
- <screen>
- YAZCONFIG=/usr/local/bin/yaz-config
- CFLAGS=`$(YAZCONFIG) --cflags`
- LIBS=`$(YAZCONFIG) --libs`
- myprog: myprog.o
- $(CC) $(CFLAGS) -o myprog myprog.o $(LIBS)
- </screen>
- </para>
- <para>
- The CFLAGS variable consists of a C compiler directive that will set
- the include path to the <emphasis>parent</emphasis> directory
- of <filename>yaz</filename>. That is, if &yaz; header files were
- installed in <filename>/usr/local/include/yaz</filename>,
- then include path is set to <filename>/usr/local/include</filename>.
- Therefore, in your applications you should use
- <screen>
- #include <yaz/proto.h>
- </screen>
- and <emphasis>not</emphasis>
- <screen>
- #include <proto.h>
- </screen>
- </para>
- <para>
- For Libtool users, the <filename>yaz-config</filename> script provides
- a different variant of option <literal>--libs</literal>, called
- <literal>--lalibs</literal> that returns the name of the
- Libtool archive(s) for &yaz; rather than the ordinary ones.
- </para>
- <para>
- For applications using the threaded version of &yaz;,
- specify <literal>threads</literal> after the
- other options. When <literal>threads</literal> is given,
- more flags and linker flags will be printed by
- <filename>yaz-config</filename>. If our previous example was
- using threads, you'd have to modify the lines that set
- <literal>CFLAGS</literal> and <literal>LIBS</literal> as
- follows:
- <screen>
- CFLAGS=`$(YAZCONFIG) --cflags threads`
- LIBS=`$(YAZCONFIG) --libs threads`
- </screen>
- There is no need specify POSIX thread libraries in your Makefile.
- The <literal>LIBS</literal> variable includes that as well.
- </para>
- </sect2>
- </sect1>
- <sect1 id="installation.win32"><title>WIN32</title>
-
- <para>The easiest way to install YAZ on Windows is by downloading
- an installer from
- <ulink url="&url.yaz.download.win32;">here</ulink>.
- The installer comes with source too - in case you wish to
- compile YAZ with different compiler options, etc.
- </para>
-
- <sect2 id="installation.win32.source">
- <title>Compiling from Source on WIN32</title>
- <para>
- &yaz; is shipped with "makefiles" for the NMAKE tool that comes
- with <ulink url="&url.vstudio;">
- Microsoft Visual Studio</ulink>. It has been tested with
- Microsoft Visual Studio 2003/2005/2008.
- </para>
- <para>
- Start a command prompt and switch the sub directory
- <filename>WIN</filename> where the file <filename>makefile</filename>
- is located. Customize the installation by editing the
- <filename>makefile</filename> file (for example by using notepad).
-
- The following summarizes the most important settings in that file:
-
- <variablelist>
- <varlistentry><term><literal>DEBUG</literal></term>
- <listitem><para>
- If set to 1, the software is
- compiled with debugging libraries (code generation is
- multi-threaded debug DLL).
- If set to 0, the software is compiled with release libraries
- (code generation is multi-threaded DLL).
- </para></listitem>
- </varlistentry>
- <varlistentry>
- <term><literal>HAVE_TCL</literal>, <literal>TCL</literal></term>
- <listitem><para>
- If <literal>HAVE_TCL</literal> is set to 1, nmake will
- use the ASN.1 compiler (<ulink url="&url.tcl;">Tcl</ulink> based).
- You must set <literal>TCL</literal> to the full path of the Tcl
- interpreter. A Windows version of Tcl is part of
- <ulink url="&url.gitwindows;">Git for Windows</ulink>.
- </para>
- <para>
- If you do not have Tcl installed, set
- <literal>HAVE_TCL</literal> to 0.
- </para></listitem>
- </varlistentry>
-
- <varlistentry>
- <term><literal>HAVE_BISON</literal>,
- <literal>BISON</literal></term>
- <listitem><para>
- If GNU Bison is present, you might set <literal>HAVE_BISON</literal>
- to 1 and specify the Bison executable in <literal>BISON</literal>.
- Bison is only required if you use the Git version of
- YAZ or if you modify the grammar for CQL
- (<filename>cql.y</filename>).
- </para>
- <para>
- A Windows version of GNU Bison is part of
- <ulink url="&url.gitwindows;">Git for Windows</ulink>.
- </para></listitem>
- </varlistentry>
-
- <varlistentry>
- <term><literal>HAVE_ICONV</literal>,
- <literal>ICONV_DIR</literal></term>
- <listitem><para>
- If <literal>HAVE_ICONV</literal> is set to 1, YAZ is compiled
- with iconv support. In this configuration, set
- <literal>ICONV_DIR</literal> to the iconv source directory.
- </para></listitem>
- </varlistentry>
-
- <varlistentry>
- <term><literal>HAVE_LIBXML2</literal>,
- <literal>LIBXML2_DIR</literal></term>
- <listitem>
- <para>
- If <literal>HAVE_LIBXML2</literal> is set to 1, YAZ is compiled
- with SRU support. In this configuration, set
- <literal>LIBXML2_DIR</literal> to the
- <ulink url="&url.libxml2;">libxml2</ulink> source directory
- and
- <literal>ZLIB_DIR</literal> to the zlib directory.
- </para>
-
- <para>
- Windows versions of libxslt, libxml2, zlib and iconv can be found
- <ulink url="&url.libxml2.download.win32;">
- Igor Zlatković' site</ulink>.
- </para>
-
- <note>
- <para>
- YAZ is not using zlib but libxml2 is depending on it.
- </para>
- </note>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><literal>HAVE_LIBXSLT</literal>,
- <literal>LIBXSLT_DIR</literal></term>
- <listitem>
- <para>
- If <literal>HAVE_LIBXSLT</literal> is set to 1, YAZ is compiled
- with XSLT support. In this configuration, set
- <literal>LIBXSLT_DIR</literal> to the
- <ulink url="&url.libxslt;">libxslt</ulink> source directory.
- </para>
-
- <note>
- <para>
- libxslt depends libxml2.
- </para>
- </note>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term><literal>HAVE_ICU</literal>,
- <literal>ICU_DIR</literal></term>
- <listitem>
- <para>
- If <literal>HAVE_ICU</literal> is set to 1, YAZ is compiled
- with <ulink url="&url.icu;">ICU</ulink> support.
- In this configuration, set
- <literal>ICU_DIR</literal> to the
- <ulink url="&url.icu;">ICU</ulink> source directory.
- </para>
- </listitem>
-
- </varlistentry>
-
- </variablelist>
- </para>
- <para>
- When satisfied with the settings in the makefile, type
- <screen>
- nmake
- </screen>
- </para>
- <note>
- <para>
- If the <filename>nmake</filename> command is not found on your system
- you probably haven't defined the environment variables required to
- use that tool. To fix that, find and run the batch file
- <filename>vcvars32.bat</filename>. You need to run it from within
- the command prompt or set the environment variables "globally";
- otherwise it doesn't work.
- </para>
- </note>
- <para>
- If you wish to recompile &yaz; - for example if you modify
- settings in the <filename>makefile</filename> you can delete
- object files, etc by running.
- <screen>
- nmake clean
- </screen>
- </para>
- <para>
- The following files are generated upon successful compilation:
-
- <variablelist>
- <varlistentry><term><filename>bin/yaz&soversion;.dll</filename> /
- <filename>bin/yaz&soversion;d.dll</filename></term>
- <listitem><para>
- &yaz; Release/Debug DLL.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>lib/yaz&soversion;.lib</filename> /
- <filename>lib/yaz&soversion;d.lib</filename></term>
- <listitem><para>
- Import library for <filename>yaz&soversion;.dll</filename> /
- <filename>yaz&soversion;d.dll</filename>.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>bin/yaz_cond&soversion;.dll</filename> /
- <filename>bin/yaz_cond&soversion;d.dll</filename></term>
- <listitem><para>
- Release/Debug DLL for condition variable utilities (condvar.c).
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>lib/yaz_cond&soversion;.lib</filename> /
- <filename>lib/yaz_cond&soversion;d.lib</filename></term>
- <listitem><para>
- Import library for <filename>yaz_cond&soversion;.dll</filename> /
- <filename>yaz_cond&soversion;d.dll</filename>.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>bin/yaz_icu&soversion;.dll</filename> /
- <filename>bin/yaz_icu&soversion;d.dll</filename></term>
- <listitem><para>
- Release/Debug DLL for the ICU wrapper utility.
- Only build if HAVE_ICU is 1.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>lib/yaz_icu&soversion;.lib</filename> /
- <filename>lib/yaz_icu&soversion;d.lib</filename></term>
- <listitem><para>
- Import library for <filename>yaz_icu&soversion;.dll</filename> /
- <filename>yaz_icu&soversion;d.dll</filename>.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>bin/yaz-ztest.exe</filename></term>
- <listitem><para>
- Z39.50 multi-threaded test/example server. It's a WIN32
- console application.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>bin/yaz-client.exe</filename></term>
- <listitem><para>
- &yaz; Z39.50 client application. It's a WIN32 console application.
- See chapter <link linkend="yaz-client">YAZ client</link> for more
- information.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>bin/yaz-icu.exe</filename></term>
- <listitem><para>This program exposes the ICU wrapper library if that
- is enabled for YAZ. Only if ICU is available this program is
- build.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>bin/zoomsh.exe</filename></term>
- <listitem><para>
- Simple console application implemented on top of the
- <link linkend="zoom">ZOOM</link> functions.
- The application is a command line shell that allows you to enter
- simple commands to perform ZOOM operations.
- </para></listitem></varlistentry>
-
- <varlistentry><term><filename>bin/zoomtst1.exe</filename>,
- <filename>bin/zoomtst2.exe</filename>, ..</term>
- <listitem><para>
- Several small applications that demonstrates the ZOOM API.
- </para></listitem></varlistentry>
-
- </variablelist>
-
- </para>
- </sect2>
-
- <sect2 id="installation-linking-yaz-win32">
- <title>How to make apps using YAZ on WIN32</title>
- <para>
- This section will go though the process of linking your WIN32
- applications with &yaz;.
- </para>
- <para>
- Some people are confused by the fact that we use the nmake
- tool to build &yaz;. They think they have to do that too - in order
- to make their WIN32 applications work with &yaz;. The good news is that
- you don't have to. You can use the integrated environment of
- Visual Studio if desired for your own application.
- </para>
- <para>
- When setting up a project or Makefile you have to set the following:
- <variablelist>
- <varlistentry><term>include path</term><listitem><para>
- Set it to the <filename>include</filename> directory of &yaz;.
- </para></listitem></varlistentry>
- <varlistentry><term>import library <filename>yaz&soversion;.lib</filename>
- </term><listitem><para>
- You must link with this library. It's located in the
- sub directory <filename>lib</filename> of &yaz;.
- If you want to link with the debug version of &yaz;, you must
- link against <filename>yaz&soversion;d.lib</filename> instead.
- </para></listitem></varlistentry>
- <varlistentry><term>dynamic link library
- <filename>yaz&soversion;.dll</filename>
- </term><listitem><para>
- This DLL must be in your execution path when you invoke
- your application. Specifically, you should distribute this
- DLL with your application.
- </para></listitem></varlistentry>
- </variablelist>
- </para>
- </sect2>
- </sect1>
- </chapter>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
-
+++ /dev/null
- <chapter id="introduction"><title>Introduction</title>
-
- <para>
- &yaz; is a C/C++ library for information retrieval applications
- using the Z39.50/SRU/Solr protocols for information retrieval.
- </para>
-
- <para>
- Properties of &yaz;:
- <itemizedlist>
- <listitem><para>
- Complete
- <ulink url="&url.z39.50;">Z39.50</ulink> version 3 support.
- Amendments and Z39.50-2002 revision is supported.
- </para></listitem>
- <listitem><para>
- Supports
- <ulink url="&url.sru;">SRU GET/POST/SOAP</ulink>
- version 1.1, 1.2 and 2.0 (over HTTP and HTTPS).
- </para></listitem>
- <listitem><para>
- Includes BER encoders/decoders for the
- <ulink url="&url.ill;">ISO ILL</ulink>
- protocol.
- </para></listitem>
- <listitem><para>
- Supports
- <ulink url="&url.solr;">Solr</ulink> Web Service version 1.4.x
- (client side only)
- </para></listitem>
- <listitem><para>
- Supports the following transports: BER over TCP/IP
- (<ulink url="&url.ber.over.tcpip;">RFC1729</ulink>),
- BER over unix local socket, and
- <ulink url="&url.http.1.1;">HTTP 1.1</ulink>.
- </para></listitem>
- <listitem><para>
- Secure Socket Layer support using
- <ulink url="&url.gnutls;">GnuTLS</ulink>.
- If enabled, &yaz; uses HTTPS transport (for SOAP) or
- "Secure BER" (for Z39.50).
- </para></listitem>
- <listitem><para>
- Offers
- <ulink url="&url.zoom;">ZOOM</ulink> C API implementing
- Z39.50, SRU and Solr Web Service.
- </para></listitem>
- <listitem><para>
- The &yaz; library offers a set of useful utilities
- related to the protocols, such as MARC (ISO2709) parser,
- CCL (ISO8777) parser,
- <ulink url="&url.cql;">CQL</ulink>
- parser, memory management routines, character set conversion.
- </para></listitem>
- <listitem><para>
- Portable code. &yaz; compiles out-of-the box on most Unixes and
- on Windows using Microsoft Visual C++.
- </para></listitem>
- <listitem><para>
- Fast operation. The C based BER encoders/decoders as well
- as the server component of &yaz; is very fast.
- </para></listitem>
- <listitem><para>
- Liberal license that allows for commercial use of &yaz;.
- </para></listitem>
- </itemizedlist>
- </para>
-
- <sect1 id="introduction.reading"><title>Reading this Manual</title>
- <para>Most implementors only need to read a fraction of the
- material in thie manual, so a quick walkthrough of the chapters
- is in order.
- </para>
- <itemizedlist>
- <listitem>
- <para>
- <xref linkend="installation"/> contains installation
- instructions for &yaz;. You don't need reading this
- if you expect to download &yaz; binaries.
- However, the chapter contains information about how
- to make <emphasis>your</emphasis> application link
- with &yaz;.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <xref linkend="zoom"/> describes the ZOOM API of &yaz;.
- This is definitely worth a read if you wish to develop a Z39.50/SRU
- client.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <xref linkend="server"/> describes the generic frontend server
- and explains how to develop server Z39.50/SRU applications for &yaz;.
- Obviously worth reading if you're to develop a server.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <xref linkend="yaz-client"/> describes how to use the &yaz; Z39.50
- client. If you're developer and wish to test your server
- or a server from another party, you might find this chapter
- useful.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <xref linkend="asn"/> documents the most commonly used Z39.50
- C data structures offered by the &yaz; API. Client
- developers using ZOOM and non-Z39.50 implementors may skip this.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <xref linkend="soap"/> describes how SRU and SOAP is used
- in &yaz;. Only if you're developing SRU applications
- this section is a must.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <xref linkend="tools"/> contains sections for the various
- tools offered by &yaz;. Scan through the material quickly
- and see what's relevant to you! SRU implementors
- might find the <link linkend="cql">CQL</link> section
- particularly useful.
- </para>
- </listitem>
-
- <listitem>
- <para>
- <xref linkend="odr"/> goes through the details of the
- ODR module which is the work horse that encodes and decodes
- BER packages. Implementors using ZOOM only, do <emphasis>not</emphasis>
- need reading this.
- Most other Z39.50 implementors only need to read the first two
- sections (<xref linkend="odr.introduction"/> and
- <xref linkend="odr.use"/>).
- </para>
- </listitem>
-
- <listitem>
- <para>
- <xref linkend="comstack"/> describes the network layer module
- COMSTACK. Implementors using ZOOM or the generic frontend server
- may skip this. Others, presumably, handling client/server
- communication on their own should read this.
- </para>
- </listitem>
-
- </itemizedlist>
- </sect1>
- <sect1 id="introduction.api"><title>The API</title>
-
- <para>
- The <ulink url="&url.yaz;">&yaz;</ulink>
- toolkit offers several different levels of access to the
- <ulink url="&url.z39.50;">ISO23950/Z39.50</ulink>,
- <ulink url="&url.ill;">ILL</ulink> and
- <ulink url="&url.sru;">SRU</ulink>
- protocols.
- The level that you need to use depends on your requirements, and
- the role (server or client) that you want to implement.
- If you're developing a client application you should consider the
- <link linkend="zoom">ZOOM</link> API.
- It is, by far, the easiest way to develop clients in C.
- Server implementers should consider the
- <link linkend="server">generic frontend server</link>.
- None of those high-level APIs support the whole protocol, but
- they do include most facilities used in existing Z39.50 applications.
- </para>
- <para>
- If you're using 'exotic' functionality (meaning anything not included in
- the high-level APIs), developing non-standard extensions to Z39.50 or
- you're going to develop an ILL application you'll have to learn the lower
- level APIs of &yaz;.
- </para>
- <para>
- The YAZ toolkit modules is shown in figure <xref linkend="yaz.layer"/>.
- </para>
- <figure id="yaz.layer">
- <title>YAZ layers</title>
- <mediaobject>
- <imageobject>
- <imagedata fileref="apilayer.png" format="PNG"/>
- </imageobject>
- <imageobject>
- <imagedata fileref="apilayer.eps" format="EPS"/>
- </imageobject>
- </mediaobject>
- </figure>
- <para>
- There are four layers.
- <itemizedlist>
- <listitem>
- <para>A client or server application (or both).
- This layer includes ZOOM and the generic frontend server.
- </para>
- </listitem>
- <listitem>
- <para>
- The second layer provides a C represenation of the
- protocol units (packages) for Z39.50 ASN.1, ILL ASN.1,
- SRU.
- </para>
- </listitem>
- <listitem>
- <para>
- The third layer encodes and decodes protocol data units to
- simple packages (buffer with certain length). The &odr; module
- encodes and decodes BER whereas the HTTP modules encodes and
- decodes HTTP ruquests/responses.
- </para>
- </listitem>
- <listitem>
- <para>
- The lowest layer is &comstack; which exchanges the encoded packages
- with a peer process over a network.
- </para>
- </listitem>
- </itemizedlist>
- </para>
- <para>
- The &asn; module represents the ASN.1 definition of
- the Z39.50 protocol. It establishes a set of type and
- structure definitions, with one structure for each of the top-level
- PDUs, and one structure or type for each of the contained ASN.1 types.
- For primitive types, or other types that are defined by the ASN.1
- standard itself (such as the EXTERNAL type), the C representation is
- provided by the &odr; (Open Data Representation) subsystem.
- </para>
- <para>
- &odr; is a basic mechanism for representing an
- ASN.1 type in the C programming language, and for implementing BER
- encoders and decoders for values of that type. The types defined in
- the &asn; module generally have the prefix <literal>Z_</literal>, and
- a suffix corresponding to the name of the type in the ASN.1
- specification of the protocol (generally Z39.50-1995). In the case of
- base types (those originating in the ASN.1 standard itself), the prefix
- <literal>Odr_</literal> is sometimes seen. Either way, look for
- the actual definition in either <filename>z-core.h</filename> (for the types
- from the protocol), <filename>odr.h</filename> (for the primitive ASN.1
- types).
- The &asn; library also provides functions (which are, in turn,
- defined using &odr; primitives) for encoding and decoding data values.
- Their general form is
-
- <funcsynopsis>
- <funcprototype><funcdef>int <function>z_<replaceable>xxx</replaceable></function></funcdef>
- <paramdef>ODR <parameter>o</parameter></paramdef>
- <paramdef>Z_<replaceable>xxx</replaceable> **<parameter>p</parameter></paramdef>
- <paramdef>int <parameter>optional</parameter></paramdef>
- <paramdef>const char *<parameter>name</parameter></paramdef>
- </funcprototype>
- </funcsynopsis>
- (note the lower-case "z" in the function name)
- </para>
-
- <note>
- <para>
- If you are using the premade definitions of the &asn; module, and you
- are not adding new protocol of your own, the only parts of &odr; that you
- need to worry about are documented in
- <xref linkend="odr.use"/>.
- </para>
- </note>
-
- <para>
- When you have created a BER-encoded buffer, you can use the &comstack;
- subsystem to transmit (or receive) data over the network. The &comstack;
- module provides simple functions for establishing a connection
- (passively or actively, depending on the role of your application),
- and for exchanging BER-encoded PDUs over that connection. When you
- create a connection endpoint, you need to specify what transport to
- use (TCP/IP, SSL or UNIX sockets).
- For the remainder of the connection's lifetime, you don't have
- to worry about the underlying transport protocol at all - the &comstack;
- will ensure that the correct mechanism is used.
- </para>
- <para>
- We call the combined interfaces to &odr;, &asn;, and &comstack; the service
- level API. It's the API that most closely models the Z39.50
- service/protocol definition, and it provides unlimited access to all
- fields and facilities of the protocol definitions.
- </para>
- <para>
- The reason that the &yaz; service-level API is a conglomerate of the
- APIs from three different submodules is twofold. First, we wanted to allow
- the user a choice of different options for each major task. For instance,
- if you don't like the protocol API provided by &odr;/&asn;, you
- can use SNACC or BERUtils instead, and still have the benefits of the
- transparent transport approach of the &comstack; module. Secondly,
- we realize that you may have to fit the toolkit into an existing
- event-processing structure, in a way that is incompatible with
- the &comstack; interface or some other part of &yaz;.
- </para>
- </sect1>
- </chapter>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document:"yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
+++ /dev/null
- <appendix id="license"><title>License</title>
-
- <sect1 id="license.indexdata"><title>Index Data Copyright</title>
-
- <para>
- Copyright © ©right-year; Index Data.
- </para>
- <para>
- All rights reserved.
- </para>
-
- <para>
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions are met:
- </para>
-
- <itemizedlist>
- <listitem>
- <para>
- Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- </para>
- </listitem>
- <listitem>
- <para>
- Redistributions in binary form must reproduce the above copyright
- notice, this list of conditions and the following disclaimer in the
- documentation and/or other materials provided with the distribution.
- </para>
- </listitem>
- <listitem>
- <para>
- Neither the name of Index Data nor the names of its contributors
- may be used to endorse or promote products derived from this
- software without specific prior written permission.
- </para>
- </listitem>
- </itemizedlist>
-
- <para>
- THIS SOFTWARE IS PROVIDED BY INDEX DATA ``AS IS'' AND ANY
- EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
- DISCLAIMED. IN NO EVENT SHALL INDEX DATA BE LIABLE FOR
- ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
- DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
- SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
- CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
- LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
- OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
- SUCH DAMAGE.
- </para>
- </sect1>
- </appendix>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
+++ /dev/null
- <chapter id="odr"><title>The ODR Module</title>
-
- <sect1 id="odr.introduction"><title>Introduction</title>
-
- <para>
- &odr; is the BER-encoding/decoding subsystem of &yaz;. Care as been taken
- to isolate &odr; from the rest of the package - specifically from the
- transport interface. &odr; may be used in any context where basic
- ASN.1/BER representations are used.
- </para>
-
- <para>
- If you are only interested in writing a Z39.50 implementation based on
- the PDUs that are already provided with &yaz;, you only need to concern
- yourself with the section on managing ODR streams
- (<xref linkend="odr.use"/>). Only if you need to
- implement ASN.1 beyond that which has been provided, should you
- worry about the second half of the documentation
- (<xref linkend="odr.programming"/>).
- If you use one of the higher-level interfaces, you can skip this
- section entirely.
- </para>
-
- <para>
- This is important, so we'll repeat it for emphasis: <emphasis>You do
- not need to read <xref linkend="odr.programming"/>
- to implement Z39.50 with &yaz;.</emphasis>
- </para>
-
- <para>
- If you need a part of the protocol that isn't already in &yaz;, you
- should contact the authors before going to work on it yourself: We
- might already be working on it. Conversely, if you implement a useful
- part of the protocol before us, we'd be happy to include it in a
- future release.
- </para>
-
- </sect1>
- <sect1 id="odr.use"><title>Using ODR</title>
-
- <sect2 id="odr.streams"><title>ODR Streams</title>
-
- <para>
- Conceptually, the ODR stream is the source of encoded data in the
- decoding mode; when encoding, it is the receptacle for the encoded
- data. Before you can use an ODR stream it must be allocated. This is
- done with the function
- </para>
-
- <synopsis>
- ODR odr_createmem(int direction);
- </synopsis>
-
- <para>
- The <function>odr_createmem()</function> function takes as argument one
- of three manifest constants: <literal>ODR_ENCODE</literal>,
- <literal>ODR_DECODE</literal>, or <literal>ODR_PRINT</literal>.
- An &odr; stream can be in only one mode - it is not possible to change
- its mode once it's selected. Typically, your program will allocate
- at least two ODR streams - one for decoding, and one for encoding.
- </para>
-
- <para>
- When you're done with the stream, you can use
- </para>
-
- <synopsis>
- void odr_destroy(ODR o);
- </synopsis>
-
- <para>
- to release the resources allocated for the stream.
- </para>
- </sect2>
-
- <sect2 id="odr.memory.management"><title id="memory">Memory Management</title>
-
- <para>
- Two forms of memory management take place in the &odr; system. The first
- one, which has to do with allocating little bits of memory (sometimes
- quite large bits of memory, actually) when a protocol package is
- decoded, and turned into a complex of interlinked structures. This
- section deals with this system, and how you can use it for your own
- purposes. The next section deals with the memory management which is
- required when encoding data - to make sure that a large enough buffer is
- available to hold the fully encoded PDU.
- </para>
-
- <para>
- The &odr; module has its own memory management system, which is
- used whenever memory is required. Specifically, it is used to allocate
- space for data when decoding incoming PDUs. You can use the memory
- system for your own purposes, by using the function
- </para>
-
- <synopsis>
- void *odr_malloc(ODR o, size_t size);
- </synopsis>
-
- <para>
- You can't use the normal <function>free(2)</function> routine to free
- memory allocated by this function, and &odr; doesn't provide a parallel
- function. Instead, you can call
- </para>
-
- <synopsis>
- void odr_reset(ODR o);
- </synopsis>
-
- <para>
- when you are done with the
- memory: Everything allocated since the last call to
- <function>odr_reset()</function> is released.
- The <function>odr_reset()</function> call is also required to clear
- up an error condition on a stream.
- </para>
-
- <para>
- The function
- </para>
-
- <synopsis>
- size_t odr_total(ODR o);
- </synopsis>
-
- <para>
- returns the number of bytes allocated on the stream since the last call to
- <function>odr_reset()</function>.
- </para>
-
- <para>
- The memory subsystem of &odr; is fairly efficient at allocating and
- releasing little bits of memory. Rather than managing the individual,
- small bits of space, the system maintains a free-list of larger chunks
- of memory, which are handed out in small bits. This scheme is
- generally known as a <emphasis>nibble memory</emphasis> system.
- It is very useful for maintaining short-lived constructions such
- as protocol PDUs.
- </para>
-
- <para>
- If you want to retain a bit of memory beyond the next call to
- <function>odr_reset()</function>, you can use the function
- </para>
-
- <synopsis>
- ODR_MEM odr_extract_mem(ODR o);
- </synopsis>
-
- <para>
- This function will give you control of the memory recently allocated
- on the ODR stream. The memory will live (past calls to
- <function>odr_reset()</function>), until you call the function
- </para>
-
- <synopsis>
- void odr_release_mem(ODR_MEM p);
- </synopsis>
-
- <para>
- The opaque <literal>ODR_MEM</literal> handle has no other purpose than
- referencing the memory block for you until you want to release it.
- </para>
-
- <para>
- You can use <function>odr_extract_mem()</function> repeatedly between
- allocating data, to retain individual control of separate chunks of data.
- </para>
-
- </sect2>
- <sect2 id="odr.encoding.and.decoding"><title>Encoding and Decoding Data</title>
-
- <para>
- When encoding data, the ODR stream will write the encoded octet string
- in an internal buffer. To retrieve the data, use the function
- </para>
-
- <synopsis>
- char *odr_getbuf(ODR o, int *len, int *size);
- </synopsis>
-
- <para>
- The integer pointed to by len is set to the length of the encoded
- data, and a pointer to that data is returned. <literal>*size</literal>
- is set to the size of the buffer (unless <literal>size</literal> is null,
- signaling that you are not interested in the size). The next call to
- a primitive function using the same &odr; stream will overwrite the
- data, unless a different buffer has been supplied using the call
- </para>
-
- <synopsis>
- void odr_setbuf(ODR o, char *buf, int len, int can_grow);
- </synopsis>
-
- <para>
- which sets the encoding (or decoding) buffer used by
- <literal>o</literal> to <literal>buf</literal>, using the length
- <literal>len</literal>.
- Before a call to an encoding function, you can use
- <function>odr_setbuf()</function> to provide the stream with an encoding
- buffer of sufficient size (length). The <literal>can_grow</literal>
- parameter tells the encoding &odr; stream whether it is allowed to use
- <function>realloc(2)</function> to increase the size of the buffer when
- necessary. The default condition of a new encoding stream is equivalent
- to the results of calling
- </para>
-
- <synopsis>
- odr_setbuf(stream, 0, 0, 1);
- </synopsis>
-
- <para>
- In this case, the stream will allocate and reallocate memory as
- necessary. The stream reallocates memory by repeatedly doubling the
- size of the buffer - the result is that the buffer will typically
- reach its maximum, working size with only a small number of reallocation
- operations. The memory is freed by the stream when the latter is destroyed,
- unless it was assigned by the user with the <literal>can_grow</literal>
- parameter set to zero (in this case, you are expected to retain
- control of the memory yourself).
- </para>
-
- <para>
- To assume full control of an encoded buffer, you must first call
- <function>odr_getbuf()</function> to fetch the buffer and its length.
- Next, you should call <function>odr_setbuf()</function> to provide a
- different buffer (or a null pointer) to the stream. In the simplest
- case, you will reuse the same buffer over and over again, and you
- will just need to call <function>odr_getbuf()</function> after each
- encoding operation to get the length and address of the buffer.
- Note that the stream may reallocate the buffer during an encoding
- operation, so it is necessary to retrieve the correct address after
- each encoding operation.
- </para>
-
- <para>
- It is important to realize that the ODR stream will not release this
- memory when you call <function>odr_reset()</function>: It will
- merely update its internal pointers to prepare for the encoding of a
- new data value.
- When the stream is released by the <function>odr_destroy()</function>
- function, the memory given to it by <function>odr_setbuf</function> will
- be released <emphasis>only</emphasis> if the <literal>can_grow</literal>
- parameter to <function>odr_setbuf()</function> was nonzero. The
- <literal>can_grow</literal> parameter, in other words, is a way of
- signaling who is to own the buffer, you or the ODR stream. If you never call
- <function>odr_setbuf()</function> on your encoding stream, which is
- typically the case, the buffer allocated by the stream will belong to
- the stream by default.
- </para>
-
- <para>
- When you wish to decode data, you should first call
- <function>odr_setbuf()</function>, to tell the decoding stream
- where to find the encoded data, and how long the buffer is
- (the <literal>can_grow</literal> parameter is ignored by a decoding
- stream). After this, you can call the function corresponding to the
- data you wish to decode (eg, <function>odr_integer()</function> odr
- <function>z_APDU()</function>).
- </para>
-
- <example id="example.odr.encoding.and.decoding.functions">
- <title>Encoding and decoding functions</title>
- <synopsis>
- int odr_integer(ODR o, Odr_int **p, int optional, const char *name);
-
- int z_APDU(ODR o, Z_APDU **p, int optional, const char *name);
- </synopsis>
- </example>
-
- <para>
- If the data is absent (or doesn't match the tag corresponding to
- the type), the return value will be either 0 or 1 depending on the
- <literal>optional</literal> flag. If <literal>optional</literal>
- is 0 and the data is absent, an error flag will be raised in the
- stream, and you'll need to call <function>odr_reset()</function> before
- you can use the stream again. If <literal>optional</literal> is
- nonzero, the pointer <emphasis>pointed</emphasis> to/ by
- <literal>p</literal> will be set to the null value, and the function
- will return 1.
- The <literal>name</literal> argument is used to pretty-print the
- tag in question. It may be set to <literal>NULL</literal> if
- pretty-printing is not desired.
- </para>
-
- <para>
- If the data value is found where it's expected, the pointer
- <emphasis>pointed to</emphasis> by the <literal>p</literal> argument
- will be set to point to the decoded type.
- The space for the type will be allocated and owned by the &odr;
- stream, and it will live until you call
- <function>odr_reset()</function> on the stream. You cannot use
- <function>free(2)</function> to release the memory.
- You can decode several data elements (by repeated calls to
- <function>odr_setbuf()</function> and your decoding function), and
- new memory will be allocated each time. When you do call
- <function>odr_reset()</function>, everything decoded since the
- last call to <function>odr_reset()</function> will be released.
- </para>
-
- <example id="example.odr.encoding.of.integer">
- <title>Encoding and decoding of an integer</title>
- <para>
- The use of the double indirection can be a little confusing at first
- (its purpose will become clear later on, hopefully),
- so an example is in order. We'll encode an integer value, and
- immediately decode it again using a different stream. A useless, but
- informative operation.
- </para>
- <programlisting><![CDATA[
-void do_nothing_useful(Odr_int value)
-{
- ODR encode, decode;
- Odr_int *valp, *resvalp;
- char *bufferp;
- int len;
-
- /* allocate streams */
- if (!(encode = odr_createmem(ODR_ENCODE)))
- return;
- if (!(decode = odr_createmem(ODR_DECODE)))
- return;
-
- valp = &value;
- if (odr_integer(encode, &valp, 0, 0) == 0)
- {
- printf("encoding went bad\n");
- return;
- }
- bufferp = odr_getbuf(encode, &len, 0);
- printf("length of encoded data is %d\n", len);
-
- /* now let's decode the thing again */
- odr_setbuf(decode, bufferp, len, 0);
- if (odr_integer(decode, &resvalp, 0, 0) == 0)
- {
- printf("decoding went bad\n");
- return;
- }
- /* ODR_INT_PRINTF format for printf (such as %d) */
- printf("the value is " ODR_INT_PRINTF "\n", *resvalp);
-
- /* clean up */
- odr_destroy(encode);
- odr_destroy(decode);
-}
-]]>
- </programlisting>
- <para>
- This looks like a lot of work, offhand. In practice, the &odr; streams
- will typically be allocated once, in the beginning of your program
- (or at the beginning of a new network session), and the encoding
- and decoding will only take place in a few, isolated places in your
- program, so the overhead is quite manageable.
- </para>
- </example>
-
- </sect2>
-
- <sect2 id="odr.printing"><title>Printing</title>
- <para>
- When an ODR stream is created of type <literal>ODR_PRINT</literal>
- the ODR module will print the contents of a PDU in a readable format.
- By default output is written to the <literal>stderr</literal> stream.
- This behavior can be changed, however, by calling the function
- <synopsis>
- odr_setprint(ODR o, FILE *file);
- </synopsis>
- before encoders or decoders are being invoked.
- It is also possible to direct the output to a buffer (of indeed
- another file), by using the more generic mechanism:
- <synopsis>
- void odr_set_stream(ODR o, void *handle,
- void (*stream_write)(ODR o, void *handle, int type,
- const char *buf, int len),
- void (*stream_close)(void *handle));
- </synopsis>
- Here the user provides an opaque handle and two handlers,
- <replaceable>stream_write</replaceable> for writing,
- and <replaceable>stream_close</replaceable> which is supposed
- to close/free resources associated with handle.
- The <replaceable>stream_close</replaceable> handler is optional and
- if NULL for the function is provided, it will not be invoked.
- The <replaceable>stream_write</replaceable> takes the ODR handle
- as parameter, the user defined handle, a type
- <literal>ODR_OCTETSTRING</literal>, <literal>ODR_VISIBLESTRING</literal>
- which indicates the type of contents is being written.
- </para>
- <para>
- Another utility useful for diagnostics (error handling) or as
- part of the printing facilities is:
- <synopsis>
- const char **odr_get_element_path(ODR o);
- </synopsis>
- which returns a list of current elements that ODR deals with at the
- moment. For the returned array, say <literal>ar</literal>,
- <literal>ar[0]</literal> is the top level element,
- <literal>ar[n]</literal> is the last. The last element has the
- property that <literal>ar[n+1] == NULL</literal>.
- </para>
- <example id="example.odr.element.path.record">
- <title>Element Path for record</title>
- <para>
- For a database record part of a PresentResponse the
- array returned by <function>odr_get_element</function>
- is <literal>presentResponse</literal>, <literal>databaseOrSurDiagnostics</literal>, <literal>?</literal>, <literal>record</literal>, <literal>?</literal>, <literal>databaseRecord</literal> . The question mark appears due to
- unnamed constructions.
- </para>
- </example>
- </sect2>
- <sect2 id="odr.diagnostics"><title>Diagnostics</title>
-
- <para>
- The encoding/decoding functions all return 0 when an error occurs.
- Until you call <function>odr_reset()</function>, you cannot use the
- stream again, and any function called will immediately return 0.
- </para>
-
- <para>
- To provide information to the programmer or administrator, the function
- </para>
-
- <synopsis>
- void odr_perror(ODR o, char *message);
- </synopsis>
-
- <para>
- is provided, which prints the <literal>message</literal> argument to
- <literal>stderr</literal> along with an error message from the stream.
- </para>
-
- <para>
- You can also use the function
- </para>
-
- <synopsis>
- int odr_geterror(ODR o);
- </synopsis>
-
- <para>
- to get the current error number from the screen. The number will be
- one of these constants:
- </para>
-
- <table frame="top" id="odr.error.codes">
- <title>ODR Error codes</title>
- <tgroup cols="2">
- <thead>
- <row>
- <entry>code</entry>
- <entry>Description</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>OMEMORY</entry><entry>Memory allocation failed.</entry>
- </row>
-
- <row>
- <entry>OSYSERR</entry><entry>A system- or library call has failed.
- The standard diagnostic variable <literal>errno</literal> should be
- examined to determine the actual error.</entry>
- </row>
-
- <row>
- <entry>OSPACE</entry><entry>No more space for encoding.
- This will only occur when the user has explicitly provided a
- buffer for an encoding stream without allowing the system to
- allocate more space.</entry>
- </row>
-
- <row>
- <entry>OREQUIRED</entry><entry>This is a common protocol error; A
- required data element was missing during encoding or decoding.</entry>
- </row>
-
- <row>
- <entry>OUNEXPECTED</entry><entry>An unexpected data element was
- found during decoding.</entry>
- </row>
-
- <row><entry>OOTHER</entry><entry>Other error. This is typically an
- indication of misuse of the &odr; system by the programmer, and also
- that the diagnostic system isn't as good as it should be, yet.</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <para>
- The character string array
- </para>
-
- <synopsis>
- char *odr_errlist[]
- </synopsis>
-
- <para>
- can be indexed by the error code to obtain a human-readable
- representation of the problem.
- </para>
-
- </sect2>
- <sect2 id="odr.summary.and.synopsis">
- <title>Summary and Synopsis</title>
-
- <synopsis>
- #include <yaz/odr.h>
-
- ODR odr_createmem(int direction);
-
- void odr_destroy(ODR o);
-
- void odr_reset(ODR o);
-
- char *odr_getbuf(ODR o, int *len, int *size);
-
- void odr_setbuf(ODR o, char *buf, int len, int can_grow);
-
- void *odr_malloc(ODR o, int size);
-
- NMEM odr_extract_mem(ODR o);
-
- int odr_geterror(ODR o);
-
- void odr_perror(ODR o, const char *message);
-
- extern char *odr_errlist[];
- </synopsis>
-
- </sect2>
- </sect1>
-
- <sect1 id="odr.programming"><title>Programming with ODR</title>
-
- <para>
- The API of &odr; is designed to reflect the structure of ASN.1, rather
- than BER itself. Future releases may be able to represent data in
- other external forms.
- </para>
-
- <tip>
- <para>
- There is an ASN.1 tutorial available at
- <ulink url="&url.asn.1.tutorial;">this site</ulink>.
- This site also has standards for ASN.1 (X.680) and BER (X.690)
- <ulink url="&url.asn.1.standards;">online</ulink>.
- </para>
- </tip>
-
- <para>
- The ODR interface is based loosely on that of the Sun Microsystems
- XDR routines.
- Specifically, each function which corresponds to an ASN.1 primitive
- type has a dual function. Depending on the settings of the ODR
- stream which is supplied as a parameter, the function may be used
- either to encode or decode data. The functions that can be built
- using these primitive functions, to represent more complex data types,
- share this quality. The result is that you only have to enter the
- definition for a type once - and you have the functionality of encoding,
- decoding (and pretty-printing) all in one unit.
- The resulting C source code is quite compact, and is a pretty
- straightforward representation of the source ASN.1 specification.
- </para>
-
- <para>
- In many cases, the model of the XDR functions works quite well in this
- role.
- In others, it is less elegant. Most of the hassle comes from the optional
- SEQUENCE members which don't exist in XDR.
- </para>
-
- <sect2 id="odr.primitive.asn1.types">
- <title>The Primitive ASN.1 Types</title>
-
- <para>
- ASN.1 defines a number of primitive types (many of which correspond
- roughly to primitive types in structured programming languages, such as C).
- </para>
-
- <sect3 id="odr.integer"><title>INTEGER</title>
-
- <para>
- The &odr; function for encoding or decoding (or printing) the ASN.1
- INTEGER type looks like this:
- </para>
-
- <synopsis>
- int odr_integer(ODR o, Odr_int **p, int optional, const char *name);
- </synopsis>
-
- <para>
- The <literal>Odr_int</literal> is just a simple integer.
- </para>
-
- <para>
- This form is typical of the primitive &odr; functions. They are named
- after the type of data that they encode or decode. They take an &odr;
- stream, an indirect reference to the type in question, and an
- <literal>optional</literal> flag (corresponding to the OPTIONAL keyword
- of ASN.1) as parameters. They all return an integer value of either one
- or zero.
- When you use the primitive functions to construct encoders for complex
- types of your own, you should follow this model as well. This
- ensures that your new types can be reused as elements in yet more
- complex types.
- </para>
-
- <para>
- The <literal>o</literal> parameter should obviously refer to a properly
- initialized &odr; stream of the right type (encoding/decoding/printing)
- for the operation that you wish to perform.
- </para>
-
- <para>
- When encoding or printing, the function first looks at
- <literal>* p</literal>. If <literal>* p</literal> (the pointer pointed
- to by <literal>p</literal>) is a null pointer, this is taken to mean that
- the data element is absent. If the <literal>optional</literal> parameter
- is nonzero, the function will return one (signifying success) without
- any further processing. If the <literal>optional</literal> is zero, an
- internal error flag is set in the &odr; stream, and the function will
- return 0. No further operations can be carried out on the stream without
- a call to the function <function>odr_reset()</function>.
- </para>
-
- <para>
- If <literal>*p</literal> is not a null pointer, it is expected to
- point to an instance of the data type. The data will be subjected to
- the encoding rules, and the result will be placed in the buffer held
- by the &odr; stream.
- </para>
-
- <para>
- The other ASN.1 primitives have similar functions that operate in
- similar manners:
- </para>
- </sect3>
- <sect3 id="odr.boolean"><title>BOOLEAN</title>
-
- <synopsis>
-int odr_bool(ODR o, Odr_bool **p, int optional, const char *name);
- </synopsis>
-
- </sect3>
- <sect3 id="odr.real"><title>REAL</title>
-
- <para>
- Not defined.
- </para>
-
- </sect3>
- <sect3 id="odr.null"><title>NULL</title>
-
- <synopsis>
-int odr_null(ODR o, Odr_null **p, int optional, const char *name);
- </synopsis>
-
- <para>
- In this case, the value of **p is not important. If <literal>*p</literal>
- is different from the null pointer, the null value is present, otherwise
- it's absent.
- </para>
-
- </sect3>
- <sect3 id="odr.octet.string"><title>OCTET STRING</title>
-
- <synopsis>
-typedef struct odr_oct
-{
- unsigned char *buf;
- int len;
-} Odr_oct;
-
-int odr_octetstring(ODR o, Odr_oct **p, int optional,
- const char *name);
- </synopsis>
-
- <para>
- The <literal>buf</literal> field should point to the character array
- that holds the octetstring. The <literal>len</literal> field holds the
- actual length.
- The character array need not be null terminated.
- </para>
-
- <para>
- To make things a little easier, an alternative is given for string
- types that are not expected to contain embedded NULL characters (eg.
- VisibleString):
- </para>
-
- <synopsis>
- int odr_cstring(ODR o, char **p, int optional, const char *name);
- </synopsis>
-
- <para>
- Which encoded or decodes between OCTETSTRING representations and
- null-terminates C strings.
- </para>
-
- <para>
- Functions are provided for the derived string types, eg:
- </para>
-
- <synopsis>
-int odr_visiblestring(ODR o, char **p, int optional,
- const char *name);
- </synopsis>
-
- </sect3>
- <sect3 id="odr.bit.string"><title>BIT STRING</title>
-
- <synopsis>
-int odr_bitstring(ODR o, Odr_bitmask **p, int optional,
- const char *name);
- </synopsis>
-
- <para>
- The opaque type <literal>Odr_bitmask</literal> is only suitable for
- holding relatively brief bit strings, eg. for options fields, etc.
- The constant <literal>ODR_BITMASK_SIZE</literal> multiplied by 8
- gives the maximum possible number of bits.
- </para>
-
- <para>
- A set of macros are provided for manipulating the
- <literal>Odr_bitmask</literal> type:
- </para>
-
- <synopsis>
-void ODR_MASK_ZERO(Odr_bitmask *b);
-
-void ODR_MASK_SET(Odr_bitmask *b, int bitno);
-
-void ODR_MASK_CLEAR(Odr_bitmask *b, int bitno);
-
-int ODR_MASK_GET(Odr_bitmask *b, int bitno);
- </synopsis>
-
- <para>
- The functions are modeled after the manipulation functions that
- accompany the <literal>fd_set</literal> type used by the
- <function>select(2)</function> call.
- <literal>ODR_MASK_ZERO</literal> should always be called first on a
- new bitmask, to initialize the bits to zero.
- </para>
- </sect3>
-
- <sect3 id="odr.object.identifier"><title>OBJECT IDENTIFIER</title>
-
- <synopsis>
-int odr_oid(ODR o, Odr_oid **p, int optional, const char *name);
- </synopsis>
-
- <para>
- The C OID representation is simply an array of integers, terminated by
- the value -1 (the <literal>Odr_oid</literal> type is synonymous with
- the <literal>short</literal> type).
- We suggest that you use the OID database module (see
- <xref linkend="tools.oid.database"/>) to handle object identifiers
- in your application.
- </para>
-
- </sect3>
- </sect2>
- <sect2 id="odr.tagging.primitive.types"><title>Tagging Primitive Types</title> <!-- tag.prim -->
-
- <para>
- The simplest way of tagging a type is to use the
- <function>odr_implicit_tag()</function> or
- <function>odr_explicit_tag()</function> macros:
- </para>
-
- <synopsis>
-int odr_implicit_tag(ODR o, Odr_fun fun, int class, int tag,
- int optional, const char *name);
-
-int odr_explicit_tag(ODR o, Odr_fun fun, int class, int tag,
- int optional, const char *name);
- </synopsis>
-
- <para>
- To create a type derived from the integer type by implicit tagging, you
- might write:
- </para>
-
- <screen>
- MyInt ::= [210] IMPLICIT INTEGER
- </screen>
-
- <para>
- In the &odr; system, this would be written like:
- </para>
-
- <screen>
-int myInt(ODR o, Odr_int **p, int optional, const char *name)
-{
- return odr_implicit_tag(o, odr_integer, p,
- ODR_CONTEXT, 210, optional, name);
-}
- </screen>
-
- <para>
- The function <function>myInt()</function> can then be used like any of
- the primitive functions provided by &odr;. Note that the behavior of
- <function>odr_explicit_tag()</function>
- and <function>odr_implicit_tag()</function> macros
- act exactly the same as the functions they are applied to - they
- respond to error conditions, etc, in the same manner - they
- simply have three extra parameters. The class parameter may
- take one of the values: <literal>ODR_CONTEXT</literal>,
- <literal>ODR_PRIVATE</literal>, <literal>ODR_UNIVERSAL</literal>, or
- <literal>/ODR_APPLICATION</literal>.
- </para>
-
- </sect2>
- <sect2 id="odr.constructed.types"><title>Constructed Types</title>
-
- <para>
- Constructed types are created by combining primitive types. The
- &odr; system only implements the SEQUENCE and SEQUENCE OF constructions
- (although adding the rest of the container types should be simple
- enough, if the need arises).
- </para>
-
- <para>
- For implementing SEQUENCEs, the functions
- </para>
-
- <synopsis>
-int odr_sequence_begin(ODR o, void *p, int size, const char *name);
-int odr_sequence_end(ODR o);
- </synopsis>
-
- <para>
- are provided.
- </para>
-
- <para>
- The <function>odr_sequence_begin()</function> function should be
- called in the beginning of a function that implements a SEQUENCE type.
- Its parameters are the &odr; stream, a pointer (to a pointer to the type
- you're implementing), and the <literal>size</literal> of the type
- (typically a C structure). On encoding, it returns 1 if
- <literal>* p</literal> is a null pointer. The <literal>size</literal>
- parameter is ignored. On decoding, it returns 1 if the type is found in
- the data stream. <literal>size</literal> bytes of memory are allocated,
- and <literal>*p</literal> is set to point to this space.
- <function>odr_sequence_end()</function> is called at the end of the
- complex function. Assume that a type is defined like this:
- </para>
-
- <screen>
-MySequence ::= SEQUENCE {
- intval INTEGER,
- boolval BOOLEAN OPTIONAL
-}
- </screen>
-
- <para>
- The corresponding &odr; encoder/decoder function and the associated data
- structures could be written like this:
- </para>
-
- <screen>
-typedef struct MySequence
-{
- Odr_int *intval;
- Odr_bool *boolval;
-} MySequence;
-
-int mySequence(ODR o, MySequence **p, int optional, const char *name)
-{
- if (odr_sequence_begin(o, p, sizeof(**p), name) == 0)
- return optional && odr_ok(o);
- return
- odr_integer(o, &(*p)->intval, 0, "intval") &&
- odr_bool(o, &(*p)->boolval, 1, "boolval") &&
- odr_sequence_end(o);
-}
-
- </screen>
-
- <para>
- Note the 1 in the call to <function>odr_bool()</function>, to mark
- that the sequence member is optional.
- If either of the member types had been tagged, the macros
- <function>odr_implicit_tag()</function> or
- <function>odr_explicit_tag()</function>
- could have been used.
- The new function can be used exactly like the standard functions provided
- with &odr;. It will encode, decode or pretty-print a data value of the
- <literal>MySequence</literal> type. We like to name types with an
- initial capital, as done in ASN.1 definitions, and to name the
- corresponding function with the first character of the name in lower case.
- You could, of course, name your structures, types, and functions any way
- you please - as long as you're consistent, and your code is easily readable.
- <literal>odr_ok</literal> is just that - a predicate that returns the
- state of the stream. It is used to ensure that the behavior of the new
- type is compatible with the interface of the primitive types.
- </para>
-
- </sect2>
- <sect2 id="odr.tagging.constructed.types">
- <title>Tagging Constructed Types</title>
-
- <note>
- <para>
- See <xref linkend="odr.tagging.primitive.types"/> for information on how to tag
- the primitive types, as well as types that are already defined.
- </para>
- </note>
-
- <sect3 id="odr.implicit.tagging">
- <title>Implicit Tagging</title>
-
- <para>
- Assume the type above had been defined as
- </para>
-
- <screen>
-MySequence ::= [10] IMPLICIT SEQUENCE {
- intval INTEGER,
- boolval BOOLEAN OPTIONAL
-}
- </screen>
-
- <para>
- You would implement this in &odr; by calling the function
- </para>
-
- <synopsis>
-int odr_implicit_settag(ODR o, int class, int tag);
- </synopsis>
-
- <para>
- which overrides the tag of the type immediately following it. The
- macro <function>odr_implicit_tag()</function> works by calling
- <function>odr_implicit_settag()</function> immediately
- before calling the function pointer argument.
- Your type function could look like this:
- </para>
-
- <screen>
-int mySequence(ODR o, MySequence **p, int optional, const char *name)
-{
- if (odr_implicit_settag(o, ODR_CONTEXT, 10) == 0 ||
- odr_sequence_begin(o, p, sizeof(**p), name) == 0)
- return optional && odr_ok(o);
- return
- odr_integer(o, &(*p)->intval, 0, "intval") &&
- odr_bool(o, &(*p)->boolval, 1, "boolval") &&
- odr_sequence_end(o);
-}
- </screen>
-
- <para>
- The definition of the structure <literal>MySequence</literal> would be
- the same.
- </para>
- </sect3>
-
- <sect3 id="odr.explicit.tagging"><title>Explicit Tagging</title>
-
- <para>
- Explicit tagging of constructed types is a little more complicated,
- since you are in effect adding a level of construction to the data.
- </para>
-
- <para>
- Assume the definition:
- </para>
-
- <screen>
-MySequence ::= [10] IMPLICIT SEQUENCE {
- intval INTEGER,
- boolval BOOLEAN OPTIONAL
-}
- </screen>
-
- <para>
- Since the new type has an extra level of construction, two new functions
- are needed to encapsulate the base type:
- </para>
-
- <synopsis>
-int odr_constructed_begin(ODR o, void *p, int class, int tag,
- const char *name);
-
-int odr_constructed_end(ODR o);
- </synopsis>
-
- <para>
- Assume that the IMPLICIT in the type definition above were replaced
- with EXPLICIT (or that the IMPLICIT keyword were simply deleted, which
- would be equivalent). The structure definition would look the same,
- but the function would look like this:
- </para>
-
- <screen>
-int mySequence(ODR o, MySequence **p, int optional, const char *name)
-{
- if (odr_constructed_begin(o, p, ODR_CONTEXT, 10, name) == 0)
- return optional && odr_ok(o);
- if (o->direction == ODR_DECODE)
- *p = odr_malloc(o, sizeof(**p));
- if (odr_sequence_begin(o, p, sizeof(**p), 0) == 0)
- {
- *p = 0; /* this is almost certainly a protocol error */
- return 0;
- }
- return
- odr_integer(o, &(*p)->intval, 0, "intval") &&
- odr_bool(o, &(*p)->boolval, 1, "boolval") &&
- odr_sequence_end(o) &&
- odr_constructed_end(o);
-}
- </screen>
-
- <para>
- Notice that the interface here gets kind of nasty. The reason is
- simple: Explicitly tagged, constructed types are fairly rare in
- the protocols that we care about, so the
- esthetic annoyance (not to mention the dangers of a cluttered
- interface) is less than the time that would be required to develop a
- better interface. Nevertheless, it is far from satisfying, and it's a
- point that will be worked on in the future. One option for you would
- be to simply apply the <function>odr_explicit_tag()</function> macro to
- the first function, and not
- have to worry about <function>odr_constructed_*</function> yourself.
- Incidentally, as you might have guessed, the
- <function>odr_sequence_</function> functions are themselves
- implemented using the <function>/odr_constructed_</function> functions.
- </para>
-
- </sect3>
- </sect2>
- <sect2 id="odr.sequence.of"><title>SEQUENCE OF</title>
-
- <para>
- To handle sequences (arrays) of a specific type, the function
- </para>
-
- <synopsis>
-int odr_sequence_of(ODR o, int (*fun)(ODR o, void *p, int optional),
- void *p, int *num, const char *name);
- </synopsis>
-
- <para>
- The <literal>fun</literal> parameter is a pointer to the decoder/encoder
- function of the type. <literal>p</literal> is a pointer to an array of
- pointers to your type. <literal>num</literal> is the number of elements
- in the array.
- </para>
-
- <para>
- Assume a type
- </para>
-
- <screen>
-MyArray ::= SEQUENCE OF INTEGER
- </screen>
-
- <para>
- The C representation might be
- </para>
-
- <screen>
-typedef struct MyArray
-{
- int num_elements;
- Odr_int **elements;
-} MyArray;
- </screen>
-
- <para>
- And the function might look like
- </para>
-
- <screen>
-int myArray(ODR o, MyArray **p, int optional, const char *name)
-{
- if (o->direction == ODR_DECODE)
- *p = odr_malloc(o, sizeof(**p));
- if (odr_sequence_of(o, odr_integer, &(*p)->elements,
- &(*p)->num_elements, name))
- return 1;
- *p = 0;
- return optional && odr_ok(o);
-}
- </screen>
-
- </sect2>
- <sect2 id="odr.choice.types"><title>CHOICE Types</title>
-
- <para>
- The choice type is used fairly often in some ASN.1 definitions, so
- some work has gone into streamlining its interface.
- </para>
-
- <para>
- CHOICE types are handled by the function:
- </para>
-
- <synopsis>
-int odr_choice(ODR o, Odr_arm arm[], void *p, void *whichp,
- const char *name);
- </synopsis>
-
- <para>
- The <literal>arm</literal> array is used to describe each of the possible
- types that the CHOICE type may assume. Internally in your application,
- the CHOICE type is represented as a discriminated union. That is, a
- C union accompanied by an integer (or enum) identifying the active
- 'arm' of the union.
- <literal>whichp</literal> is a pointer to the union discriminator.
- When encoding, it is examined to determine the current type.
- When decoding, it is set to reference the type that was found in
- the input stream.
- </para>
-
- <para>
- The Odr_arm type is defined thus:
- </para>
-
- <screen>
-typedef struct odr_arm
-{
- int tagmode;
- int class;
- int tag;
- int which;
- Odr_fun fun;
- char *name;
-} Odr_arm;
- </screen>
-
- <para>
- The interpretation of the fields are:
- </para>
-
- <variablelist>
- <varlistentry><term>tagmode</term>
- <listitem><para>Either <literal>ODR_IMPLICIT</literal>,
- <literal>ODR_EXPLICIT</literal>, or <literal>ODR_NONE</literal> (-1)
- to mark no tagging.</para></listitem>
- </varlistentry>
-
- <varlistentry><term>which</term>
- <listitem><para>The value of the discriminator that corresponds to
- this CHOICE element. Typically, it will be a #defined constant, or
- an enum member.</para></listitem>
- </varlistentry>
-
- <varlistentry><term>fun</term>
- <listitem><para>A pointer to a function that implements the type of
- the CHOICE member. It may be either a standard &odr; type or a type
- defined by yourself.</para></listitem>
- </varlistentry>
-
- <varlistentry><term>name</term>
- <listitem><para>Name of tag.</para></listitem>
- </varlistentry>
- </variablelist>
-
- <para>
- A handy way to prepare the array for use by the
- <function>odr_choice()</function> function is to
- define it as a static, initialized array in the beginning of your
- decoding/encoding function. Assume the type definition:
- </para>
-
- <screen>
-MyChoice ::= CHOICE {
- untagged INTEGER,
- tagged [99] IMPLICIT INTEGER,
- other BOOLEAN
-}
- </screen>
-
- <para>
- Your C type might look like
- </para>
-
- <screen>
-typedef struct MyChoice
-{
- enum
- {
- MyChoice_untagged,
- MyChoice_tagged,
- MyChoice_other
- } which;
- union
- {
- Odr_int *untagged;
- Odr_int *tagged;
- Odr_bool *other;
- } u;
-};
- </screen>
-
- <para>
- And your function could look like this:
- </para>
-
- <screen>
-int myChoice(ODR o, MyChoice **p, int optional, const char *name)
-{
- static Odr_arm arm[] =
- {
- {-1, -1, -1, MyChoice_untagged, odr_integer, "untagged"},
- {ODR_IMPLICIT, ODR_CONTEXT, 99, MyChoice_tagged, odr_integer,
- "tagged"},
- {-1, -1, -1, MyChoice_other, odr_boolean, "other"},
- {-1, -1, -1, -1, 0}
- };
-
- if (o->direction == ODR_DECODE)
- *p = odr_malloc(o, sizeof(**p);
- else if (!*p)
- return optional && odr_ok(o);
-
- if (odr_choice(o, arm, &(*p)->u, &(*p)->which), name)
- return 1;
- *p = 0;
- return optional && odr_ok(o);
-}
- </screen>
-
- <para>
- In some cases (say, a non-optional choice which is a member of a
- sequence), you can "embed" the union and its discriminator in the
- structure belonging to the enclosing type, and you won't need to
- fiddle with memory allocation to create a separate structure to
- wrap the discriminator and union.
- </para>
-
- <para>
- The corresponding function is somewhat nicer in the Sun XDR interface.
- Most of the complexity of this interface comes from the possibility of
- declaring sequence elements (including CHOICEs) optional.
- </para>
-
- <para>
- The ASN.1 specifications naturally requires that each member of a
- CHOICE have a distinct tag, so they can be told apart on decoding.
- Sometimes it can be useful to define a CHOICE that has multiple types
- that share the same tag. You'll need some other mechanism, perhaps
- keyed to the context of the CHOICE type. In effect, we would like to
- introduce a level of context-sensitiveness to our ASN.1 specification.
- When encoding an internal representation, we have no problem, as long
- as each CHOICE member has a distinct discriminator value. For
- decoding, we need a way to tell the choice function to look for a
- specific arm of the table. The function
- </para>
-
- <synopsis>
-void odr_choice_bias(ODR o, int what);
- </synopsis>
-
- <para>
- provides this functionality. When called, it leaves a notice for the next
- call to <function>odr_choice()</function> to be called on the decoding
- stream <literal>o</literal> that only the <literal>arm</literal> entry with
- a <literal>which</literal> field equal to <literal>what</literal>
- should be tried.
- </para>
-
- <para>
- The most important application (perhaps the only one, really) is in
- the definition of application-specific EXTERNAL encoders/decoders
- which will automatically decode an ANY member given the direct or
- indirect reference.
- </para>
-
- </sect2>
- </sect1>
-
- <sect1 id="odr.debugging"><title>Debugging</title>
-
- <para>
- The protocol modules are suffering somewhat from a lack of diagnostic
- tools at the moment. Specifically ways to pretty-print PDUs that
- aren't recognized by the system. We'll include something to this end
- in a not-too-distant release. In the meantime, what we do when we get
- packages we don't understand is to compile the ODR module with
- <literal>ODR_DEBUG</literal> defined. This causes the module to dump tracing
- information as it processes data units. With this output and the
- protocol specification (Z39.50), it is generally fairly easy to see
- what goes wrong.
- </para>
- </sect1>
- </chapter>
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
+++ /dev/null
- <chapter id="server"><title>Generic server</title>
- <sect1 id="server.introduction"><title>Introduction</title>
-
- <para>
- If you aren't into documentation, a good way to learn how the
- back end interface works is to look at the <filename>backend.h</filename>
- file. Then, look at the small dummy-server in
- <filename>ztest/ztest.c</filename>. The <filename>backend.h</filename>
- file also makes a good reference, once you've chewed your way through
- the prose of this file.
- </para>
-
- <para>
- If you have a database system that you would like to make available by
- means of Z39.50 or SRU, &yaz; basically offers your two options. You
- can use the APIs provided by the &asn;, &odr;, and &comstack;
- modules to
- create and decode PDUs, and exchange them with a client.
- Using this low-level interface gives you access to all fields and
- options of the protocol, and you can construct your server as close
- to your existing database as you like.
- It is also a fairly involved process, requiring
- you to set up an event-handling mechanism, protocol state machine,
- etc. To simplify server implementation, we have implemented a compact
- and simple, but reasonably full-functioned server-frontend that will
- handle most of the protocol mechanics, while leaving you to
- concentrate on your database interface.
- </para>
-
- <note>
- <para>
- The backend interface was designed in anticipation of a specific
- integration task, while still attempting to achieve some degree of
- generality. We realize fully that there are points where the
- interface can be improved significantly. If you have specific
- functions or parameters that you think could be useful, send us a
- mail (or better, sign on to the mailing list referred to in the
- top-level README file). We will try to fit good suggestions into future
- releases, to the extent that it can be done without requiring
- too many structural changes in existing applications.
- </para>
- </note>
-
- <note>
- <para>
- The &yaz; server does not support XCQL.
- </para>
- </note>
- </sect1>
-
- <sect1 id="server.frontend"><title>The Database Frontend</title>
-
- <para>
- We refer to this software as a generic database frontend. Your
- database system is the <emphasis>backend database</emphasis>, and the
- interface between the two is called the <emphasis>backend API</emphasis>.
- The backend API consists of a small number of function handlers and
- structure definitions. You are required to provide the
- <function>main()</function> routine for the server (which can be
- quite simple), as well as a set of handlers to match each of the
- prototypes.
- The interface functions that you write can use any mechanism you like
- to communicate with your database system: You might link the whole
- thing together with your database application and access it by
- function calls; you might use IPC to talk to a database server
- somewhere; or you might link with third-party software that handles
- the communication for you (like a commercial database client library).
- At any rate, the handlers will perform the tasks of:
- </para>
-
- <itemizedlist>
-
- <listitem><para>
- Initialization.
- </para></listitem>
-
- <listitem><para>
- Searching.
- </para></listitem>
-
- <listitem><para>
- Fetching records.
- </para></listitem>
-
- <listitem><para>
- Scanning the database index (optional - if you wish to implement SCAN).
- </para></listitem>
-
- <listitem><para>
- Extended Services (optional).
- </para></listitem>
-
- <listitem><para>
- Result-Set Delete (optional).
- </para></listitem>
-
- <listitem><para>
- Result-Set Sort (optional).
- </para></listitem>
-
- <listitem><para>
- Return Explain for SRU (optional).
- </para></listitem>
-
- </itemizedlist>
-
- <para>
- (more functions will be added in time to support as much of
- Z39.50-1995 as possible).
- </para>
-
- </sect1>
- <sect1 id="server.backend"><title>The Backend API</title>
-
- <para>
- The header file that you need to use the interface are in the
- <filename>include/yaz</filename> directory. It's called
- <filename>backend.h</filename>. It will include other files from
- the <filename>include/yaz</filename> directory, so you'll
- probably want to use the -I option of your compiler to tell it
- where to find the files. When you run
- <literal>make</literal> in the top-level &yaz; directory,
- everything you need to create your server is to link with the
- <filename>lib/libyaz.la</filename> library.
- </para>
- </sect1>
-
- <sect1 id="server.main"><title>Your main() Routine</title>
-
- <para>
- As mentioned, your <function>main()</function> routine can be quite brief.
- If you want to initialize global parameters, or read global configuration
- tables, this is the place to do it. At the end of the routine, you should
- call the function
- </para>
-
- <synopsis>
-int statserv_main(int argc, char **argv,
- bend_initresult *(*bend_init)(bend_initrequest *r),
- void (*bend_close)(void *handle));
- </synopsis>
-
- <para>
- The third and fourth arguments are pointers to handlers. Handler
- <function>bend_init</function> is called whenever the server receives
- an Initialize Request, so it serves as a Z39.50 session initializer. The
- <function>bend_close</function> handler is called when the session is
- closed.
- </para>
-
- <para>
- <function>statserv_main</function> will establish listening sockets
- according to the parameters given. When connection requests are received,
- the event handler will typically <function>fork()</function> and
- create a sub-process to handle a new connection.
- Alternatively the server may be setup to create threads for each
- connection.
- If you do use global variables and forking, you should be aware, then,
- that these cannot be shared between associations, unless you explicitly
- disable forking by command line parameters.
- </para>
-
- <para>
- The server provides a mechanism for controlling some of its behavior
- without using command-line options. The function
- </para>
-
- <synopsis>
- statserv_options_block *statserv_getcontrol(void);
- </synopsis>
-
- <para>
- will return a pointer to a <literal>struct statserv_options_block</literal>
- describing the current default settings of the server. The structure
- contains these elements:
-
- <variablelist>
- <varlistentry><term>
- <literal>int dynamic</literal></term><listitem><para>
- A boolean value, which determines whether the server
- will fork on each incoming request (TRUE), or not (FALSE). Default is
- TRUE. This flag is only read by UNIX-based servers (WIN32 based servers
- doesn't fork).
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>int threads</literal></term><listitem><para>
- A boolean value, which determines whether the server
- will create a thread on each incoming request (TRUE), or not (FALSE).
- Default is FALSE. This flag is only read by UNIX-based servers
- that offer POSIX Threads support.
- WIN32-based servers always operate in threaded mode.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>int inetd</literal></term><listitem><para>
- A boolean value, which determines whether the server
- will operates under a UNIX INET daemon (inetd). Default is FALSE.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>char logfile[ODR_MAXNAME+1]</literal></term>
- <listitem><para>File for diagnostic output ("": stderr).
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>char apdufile[ODR_MAXNAME+1]</literal></term>
- <listitem><para>
- Name of file for logging incoming and outgoing APDUs
- ("": don't log APDUs, "-":
- <literal>stderr</literal>).
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>char default_listen[1024]</literal></term>
- <listitem><para>Same form as the command-line specification of
- listener address. "": no default listener address.
- Default is to listen at "tcp:@:9999". You can only
- specify one default listener address in this fashion.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>enum oid_proto default_proto;</literal></term>
- <listitem><para>Either <literal>PROTO_Z3950</literal> or
- <literal>PROTO_SR</literal>.
- Default is <literal>PROTO_Z39_50</literal>.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>int idle_timeout;</literal></term>
- <listitem><para>Maximum session idle-time, in minutes. Zero indicates
- no (infinite) timeout. Default is 15 minutes.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>int maxrecordsize;</literal></term>
- <listitem><para>Maximum permissible record (message) size. Default
- is 64 MB. This amount of memory will only be allocated if a
- client requests a very large amount of records in one operation
- (or a big record).
- Set it to a lower number if you are worried about resource
- consumption on your host system.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>char configname[ODR_MAXNAME+1]</literal></term>
- <listitem><para>Passed to the backend when a new connection is received.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>char setuid[ODR_MAXNAME+1]</literal></term>
- <listitem><para>Set user id to the user specified, after binding
- the listener addresses.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>void (*bend_start)(struct statserv_options_block *p)</literal>
- </term>
- <listitem><para>Pointer to function which is called after the
- command line options have been parsed - but before the server
- starts listening.
- For forked UNIX servers this handler is called in the mother
- process; for threaded servers this handler is called in the
- main thread.
- The default value of this pointer is NULL in which case it
- isn't invoked by the frontend server.
- When the server operates as an NT service this handler is called
- whenever the service is started.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>void (*bend_stop)(struct statserv_options_block *p)</literal>
- </term>
- <listitem><para>Pointer to function which is called whenever the server
- has stopped listening for incoming connections. This function pointer
- has a default value of NULL in which case it isn't called.
- When the server operates as an NT service this handler is called
- whenever the service is stopped.
- </para></listitem></varlistentry>
-
- <varlistentry><term>
- <literal>void *handle</literal></term>
- <listitem><para>User defined pointer (default value NULL).
- This is a per-server handle that can be used to specify "user-data".
- Do not confuse this with the session-handle as returned by bend_init.
- </para></listitem></varlistentry>
-
- </variablelist>
- </para>
-
- <para>
- The pointer returned by <literal>statserv_getcontrol</literal> points to
- a static area. You are allowed to change the contents of the structure,
- but the changes will not take effect before you call
- </para>
-
- <synopsis>
-void statserv_setcontrol(statserv_options_block *block);
- </synopsis>
-
- <note>
- <para>
- that you should generally update this structure before calling
- <function>statserv_main()</function>.
- </para>
- </note>
- </sect1>
-
- <sect1 id="server.backendfunctions"><title>The Backend Functions</title>
-
- <para>
- For each service of the protocol, the backend interface declares one or
- two functions. You are required to provide implementations of the
- functions representing the services that you wish to implement.
- </para>
-
- <sect2 id="server.init"><title>Init</title>
-
- <synopsis>
-bend_initresult (*bend_init)(bend_initrequest *r);
- </synopsis>
-
- <para>
- This handler is called once for each new connection request, after
- a new process/thread has been created, and an Initialize Request has
- been received from the client. The pointer to the
- <function>bend_init</function> handler is passed in the call to
- <function>statserv_start</function>.
- </para>
-
- <para>
- This handler is also called when operating in SRU mode - when
- a connection has been made (even though SRU does not offer
- this service).
- </para>
-
- <para>
- Unlike previous versions of YAZ, the <function>bend_init</function> also
- serves as a handler that defines the Z39.50 services that the backend
- wish to support. Pointers to <emphasis>all</emphasis> service handlers,
- including search - and fetch must be specified here in this handler.
- </para>
- <para>
- The request - and result structures are defined as
- </para>
-
- <synopsis>
-typedef struct bend_initrequest
-{
- /** \brief user/name/password to be read */
- Z_IdAuthentication *auth;
- /** \brief encoding stream (for results) */
- ODR stream;
- /** \brief printing stream */
- ODR print;
- /** \brief decoding stream (use stream for results) */
- ODR decode;
- /** \brief reference ID */
- Z_ReferenceId *referenceId;
- /** \brief peer address of client */
- char *peer_name;
-
- /** \brief character set and language negotiation
-
- see include/yaz/z-charneg.h
- */
- Z_CharSetandLanguageNegotiation *charneg_request;
-
- /** \brief character negotiation response */
- Z_External *charneg_response;
-
- /** \brief character set (encoding) for query terms
-
- This is NULL by default. It should be set to the native character
- set that the backend assumes for query terms */
- char *query_charset;
-
- /** \brief whehter query_charset also applies to recors
-
- Is 0 (No) by default. Set to 1 (yes) if records is in the same
- character set as queries. If in doubt, use 0 (No).
- */
- int records_in_same_charset;
-
- char *implementation_id;
- char *implementation_name;
- char *implementation_version;
-
- /** \brief Z39.50 sort handler */
- int (*bend_sort)(void *handle, bend_sort_rr *rr);
- /** \brief SRU/Z39.50 search handler */
- int (*bend_search)(void *handle, bend_search_rr *rr);
- /** \brief SRU/Z39.50 fetch handler */
- int (*bend_fetch)(void *handle, bend_fetch_rr *rr);
- /** \brief SRU/Z39.50 present handler */
- int (*bend_present)(void *handle, bend_present_rr *rr);
- /** \brief Z39.50 extended services handler */
- int (*bend_esrequest) (void *handle, bend_esrequest_rr *rr);
- /** \brief Z39.50 delete result set handler */
- int (*bend_delete)(void *handle, bend_delete_rr *rr);
- /** \brief Z39.50 scan handler */
- int (*bend_scan)(void *handle, bend_scan_rr *rr);
- /** \brief Z39.50 segment facility handler */
- int (*bend_segment)(void *handle, bend_segment_rr *rr);
- /** \brief SRU explain handler */
- int (*bend_explain)(void *handle, bend_explain_rr *rr);
- /** \brief SRU scan handler */
- int (*bend_srw_scan)(void *handle, bend_scan_rr *rr);
- /** \brief SRU record update handler */
- int (*bend_srw_update)(void *handle, bend_update_rr *rr);
-
- /** \brief whether named result sets are supported (0=disable, 1=enable) */
- int named_result_sets;
-} bend_initrequest;
-
-typedef struct bend_initresult
-{
- int errcode; /* 0==OK */
- char *errstring; /* system error string or NULL */
- void *handle; /* private handle to the backend module */
-} bend_initresult;
- </synopsis>
-
- <para>
- In general, the server frontend expects that the
- <literal>bend_*result</literal> pointer that you return is valid at
- least until the next call to a <literal>bend_* function</literal>.
- This applies to all of the functions described herein. The parameter
- structure passed to you in the call belongs to the server frontend, and
- you should not make assumptions about its contents after the current
- function call has completed. In other words, if you want to retain any
- of the contents of a request structure, you should copy them.
- </para>
-
- <para>
- The <literal>errcode</literal> should be zero if the initialization of
- the backend went well. Any other value will be interpreted as an error.
- The <literal>errstring</literal> isn't used in the current version, but
- one option would be to stick it in the initResponse as a VisibleString.
- The <literal>handle</literal> is the most important parameter. It should
- be set to some value that uniquely identifies the current session to
- the backend implementation. It is used by the frontend server in any
- future calls to a backend function.
- The typical use is to set it to point to a dynamically allocated state
- structure that is private to your backend module.
- </para>
-
- <para>
- The <literal>auth</literal> member holds the authentication information
- part of the Z39.50 Initialize Request. Interpret this if your serves
- requires authentication.
- </para>
-
- <para>
- The members <literal>peer_name</literal>,
- <literal>implementation_id</literal>,
- <literal>implementation_name</literal> and
- <literal>implementation_version</literal> holds
- DNS of client, ID of implementor, name
- of client (Z39.50) implementation - and version.
- </para>
-
- <para>
- The <literal>bend_</literal> - members are set to NULL when
- <function>bend_init</function> is called. Modify the pointers by
- setting them to point to backend functions.
- </para>
-
- </sect2>
-
- <sect2 id="server.search.retrieve"><title>Search and Retrieve</title>
-
- <para>We now describe the handlers that are required to support search -
- and retrieve. You must support two functions - one for search - and one
- for fetch (retrieval of one record). If desirable you can provide a
- third handler which is called when a present request is received which
- allows you to optimize retrieval of multiple-records.
- </para>
-
- <synopsis>
-int (*bend_search) (void *handle, bend_search_rr *rr);
-
-typedef struct {
- char *setname; /* name to give to this set */
- int replace_set; /* replace set, if it already exists */
- int num_bases; /* number of databases in list */
- char **basenames; /* databases to search */
- Z_ReferenceId *referenceId;/* reference ID */
- Z_Query *query; /* query structure */
- ODR stream; /* encode stream */
- ODR decode; /* decode stream */
- ODR print; /* print stream */
-
- bend_request request;
- bend_association association;
- int *fd;
- int hits; /* number of hits */
- int errcode; /* 0==OK */
- char *errstring; /* system error string or NULL */
- Z_OtherInformation *search_info; /* additional search info */
- char *srw_sortKeys; /* holds SRU/SRW sortKeys info */
- char *srw_setname; /* holds SRU/SRW generated resultsetID */
- int *srw_setnameIdleTime; /* holds SRU/SRW life-time */
- int estimated_hit_count; /* if hit count is estimated */
- int partial_resultset; /* if result set is partial */
-} bend_search_rr;
- </synopsis>
-
- <para>
- The <function>bend_search</function> handler is a fairly close
- approximation of a protocol Z39.50 Search Request - and Response PDUs
- The <literal>setname</literal> is the resultSetName from the protocol.
- You are required to establish a mapping between the set name and whatever
- your backend database likes to use.
- Similarly, the <literal>replace_set</literal> is a boolean value
- corresponding to the resultSetIndicator field in the protocol.
- <literal>num_bases/basenames</literal> is a length of/array of character
- pointers to the database names provided by the client.
- The <literal>query</literal> is the full query structure as defined in
- the protocol ASN.1 specification.
- It can be either of the possible query types, and it's up to you to
- determine if you can handle the provided query type.
- Rather than reproduce the C interface here, we'll refer you to the
- structure definitions in the file
- <filename>include/yaz/z-core.h</filename>. If you want to look at the
- attributeSetId OID of the RPN query, you can either match it against
- your own internal tables, or you can use the <link linkend="tools.oid">
- OID tools</link>.
- </para>
-
- <para>
- The structure contains a number of hits, and an
- <literal>errcode/errstring</literal> pair. If an error occurs
- during the search, or if you're unhappy with the request, you should
- set the errcode to a value from the BIB-1 diagnostic set. The value
- will then be returned to the user in a nonsurrogate diagnostic record
- in the response. The <literal>errstring</literal>, if provided, will
- go in the addinfo field. Look at the protocol definition for the
- defined error codes, and the suggested uses of the addinfo field.
- </para>
-
- <para>
- The <function>bend_search</function> handler is also called when
- the frontend server receives a SRU SearchRetrieveRequest.
- For SRU, a CQL query is usually provided by the client.
- The CQL query is available as part of <literal>Z_Query</literal>
- structure (note that CQL is now part of Z39.50 via an external).
- To support CQL in existing implementations that only do Type-1,
- we refer to the CQL-to-PQF tool described
- <link linkend="cql.to.pqf">here</link>.
- </para>
-
- <para>
- To maintain backwards compatibility, the frontend server
- of yaz always assume that error codes are BIB-1 diagnostics.
- For SRU operation, a Bib-1 diagnostic code is mapped to
- SRU diagnostic.
- </para>
-
- <synopsis>
-int (*bend_fetch) (void *handle, bend_fetch_rr *rr);
-
-typedef struct bend_fetch_rr {
- char *setname; /* set name */
- int number; /* record number */
- Z_ReferenceId *referenceId;/* reference ID */
- Odr_oid *request_format; /* format, transfer syntax (OID) */
- Z_RecordComposition *comp; /* Formatting instructions */
- ODR stream; /* encoding stream - memory source if req */
- ODR print; /* printing stream */
-
- char *basename; /* name of database that provided record */
- int len; /* length of record or -1 if structured */
- char *record; /* record */
- int last_in_set; /* is it? */
- Odr_oid *output_format; /* response format/syntax (OID) */
- int errcode; /* 0==success */
- char *errstring; /* system error string or NULL */
- int surrogate_flag; /* surrogate diagnostic */
- char *schema; /* string record schema input/output */
-} bend_fetch_rr;
- </synopsis>
-
- <para>
- The frontend server calls the <function>bend_fetch</function> handler
- when it needs database records to fulfill a Z39.50 Search Request, a
- Z39.50 Present Request or a SRU SearchRetrieveRequest.
- The <literal>setname</literal> is simply the name of the result set
- that holds the reference to the desired record.
- The <literal>number</literal> is the offset into the set (with 1
- being the first record in the set). The <literal>format</literal> field
- is the record format requested by the client (See
- <xref linkend="tools.oid"/>).
- A value of NULL for <literal>format</literal> indicates that the
- client did not request a specific format.
- The <literal>stream</literal> argument is an &odr; stream which
- should be used for allocating space for structured data records.
- The stream will be reset when all records have been assembled, and
- the response package has been transmitted.
- For unstructured data, the backend is responsible for maintaining a
- static or dynamic buffer for the record between calls.
- </para>
-
- <para>
- If a SRU SearchRetrieveRequest is received by the frontend server,
- the <literal>referenceId</literal> is NULL and the
- <literal>format</literal> (transfer syntax) is the OID for XML.
- The schema for SRU is stored in both the
- <literal>Z_RecordComposition</literal>
- structure and <literal>schema</literal> (simple string).
- </para>
-
- <para>
- In the structure, the <literal>basename</literal> is the name of the
- database that holds the
- record. <literal>len</literal> is the length of the record returned, in
- bytes, and <literal>record</literal> is a pointer to the record.
- <literal>last_in_set</literal> should be nonzero only if the record
- returned is the last one in the given result set.
- <literal>errcode</literal> and <literal>errstring</literal>, if
- given, will be interpreted as a global error pertaining to the
- set, and will be returned in a non-surrogate-diagnostic.
- If you wish to return the error as a surrogate-diagnostic
- (local error) you can do this by setting
- <literal>surrogate_flag</literal> to 1 also.
- </para>
-
- <para>
- If the <literal>len</literal> field has the value -1, then
- <literal>record</literal> is assumed to point to a constructed data
- type. The <literal>format</literal> field will be used to determine
- which encoder should be used to serialize the data.
- </para>
-
- <note>
- <para>
- If your backend generates structured records, it should use
- <function>odr_malloc()</function> on the provided stream for allocating
- data: This allows the frontend server to keep track of the record sizes.
- </para>
- </note>
-
- <para>
- The <literal>format</literal> field is mapped to an object identifier
- in the direct reference of the resulting EXTERNAL representation
- of the record.
- </para>
-
- <note>
- <para>
- The current version of &yaz; only supports the direct reference mode.
- </para>
- </note>
-
- <synopsis>
-int (*bend_present) (void *handle, bend_present_rr *rr);
-
-typedef struct {
- char *setname; /* set name */
- int start;
- int number; /* record number */
- Odr_oid *format; /* format, transfer syntax (OID) */
- Z_ReferenceId *referenceId;/* reference ID */
- Z_RecordComposition *comp; /* Formatting instructions */
- ODR stream; /* encoding stream - memory source if required */
- ODR print; /* printing stream */
- bend_request request;
- bend_association association;
-
- int hits; /* number of hits */
- int errcode; /* 0==OK */
- char *errstring; /* system error string or NULL */
-} bend_present_rr;
- </synopsis>
-
- <para>
- The <function>bend_present</function> handler is called when
- the server receives a Z39.50 Present Request.
- The <literal>setname</literal>,
- <literal>start</literal> and <literal>number</literal> is the
- name of the result set - start position - and number of records to
- be retrieved respectively. <literal>format</literal> and
- <literal>comp</literal> is the preferred transfer syntax and element
- specifications of the present request.
- </para>
- <para>
- Note that this is handler serves as a supplement for
- <function>bend_fetch</function> and need not to be defined in order to
- support search - and retrieve.
- </para>
-
- </sect2>
-
- <sect2 id="server.delete"><title>Delete</title>
-
- <para>
- For back-ends that supports delete of a result set only one handler
- must be defined.
- </para>
-
- <synopsis>
-int (*bend_delete)(void *handle, bend_delete_rr *rr);
-
-typedef struct bend_delete_rr {
- int function;
- int num_setnames;
- char **setnames;
- Z_ReferenceId *referenceId;
- int delete_status; /* status for the whole operation */
- int *statuses; /* status each set - indexed as setnames */
- ODR stream;
- ODR print;
-} bend_delete_rr;
- </synopsis>
-
- <note>
- <para>
- The delete set function definition is rather primitive, mostly because
- we have had no practical need for it as of yet. If someone wants
- to provide a full delete service, we'd be happy to add the
- extra parameters that are required. Are there clients out there
- that will actually delete sets they no longer need?
- </para>
- </note>
-
- </sect2>
-
- <sect2 id="server.scan"><title>Scan</title>
-
- <para>
- For servers that wish to offer the scan service one handler
- must be defined.
- </para>
-
- <synopsis>
-int (*bend_scan)(void *handle, bend_scan_rr *rr);
-
-typedef enum {
- BEND_SCAN_SUCCESS, /* ok */
- BEND_SCAN_PARTIAL /* not all entries could be found */
-} bend_scan_status;
-
-typedef struct bend_scan_rr {
- int num_bases; /* number of elements in databaselist */
- char **basenames; /* databases to search */
- Odr_oid *attributeset;
- Z_ReferenceId *referenceId; /* reference ID */
- Z_AttributesPlusTerm *term;
- ODR stream; /* encoding stream - memory source if required */
- ODR print; /* printing stream */
-
- int *step_size; /* step size */
- int term_position; /* desired index of term in result list/returned */
- int num_entries; /* number of entries requested/returned */
-
- /* scan term entries. The called handler does not have
- to allocate this. Size of entries is num_entries (see above) */
- struct scan_entry *entries;
- bend_scan_status status;
- int errcode;
- char *errstring;
- char *scanClause; /* CQL scan clause */
- char *setname; /* Scan in result set (NULL if omitted) */
-} bend_scan_rr;
- </synopsis>
- <para>
- This backend server handles both Z39.50 scan
- and SRU scan. In order for a handler to distinguish between SRU (CQL) scan
- Z39.50 Scan , it must check for a non-NULL value of
- <literal>scanClause</literal>.
- </para>
- <note>
- <para>
- if designed today, it would be a choice using a union or similar,
- but that would break binary compatibility with existing servers.
- </para>
- </note>
- </sect2>
- </sect1>
-
- <sect1 id="server.invocation"><title>Application Invocation</title>
-
- <para>
- The finished application has the following
- invocation syntax (by way of <function>statserv_main()</function>):
- </para>
-
- &gfs-synopsis;
-
- <para>
- The options are:
-
- &gfs-options;
-
- </para>
-
- <para>
- A listener specification consists of a transport mode followed by a
- colon (:) followed by a listener address. The transport mode is
- either <literal>tcp</literal>, <literal>unix:</literal> or
- <literal>ssl</literal>.
- </para>
-
- <para>
- For TCP and SSL, an address has the form
- </para>
-
- <synopsis>
- hostname | IP-number [: portnumber]
- </synopsis>
-
- <para>
- The port number defaults to 210 (standard Z39.50 port).
- </para>
-
- <para>
- For UNIX, the address is the filename of socket.
- </para>
-
- <para>
- For TCP/IP and SSL, the special hostnames <literal>@</literal>,
- maps to <literal>IN6ADDR_ANY_INIT</literal> with
- IPV4 binding as well (bindv6only=0),
- The special hostname <literal>@4</literal> binds to
- <literal>INADDR_ANY</literal> (IPV4 only listener).
- The special hostname <literal>@6</literal> binds to
- <literal>IN6ADDR_ANY_INIT</literal> with bindv6only=1 (IPV6 only listener).
- </para>
-
- <example id="server.example.running.unix"><title>Running the GFS on Unix</title>
- <para>
- Assuming the server application <replaceable>appname</replaceable> is
- started as root, the following will make it listen on port 210.
- The server will change identity to <literal>nobody</literal>
- and write its log to <filename>/var/log/app.log</filename>.
- <screen>
- application -l /var/log/app.log -u nobody tcp:@:210
- </screen>
- </para>
- <para>
- The server will accept Z39.50 requests and offer SRU service on port 210.
- </para>
- </example>
- <example id="server.example.apache.sru"><title>Setting up Apache as SRU Frontend</title>
- <para>
- If you use <ulink url="&url.apache;">Apache</ulink>
- as your public web server and want to offer HTTP port 80
- access to the YAZ server on 210, you can use the
- <ulink url="&url.apache.directive.proxypass;">
- <literal>ProxyPass</literal></ulink>
- directive.
- If you have virtual host
- <literal>srw.mydomain</literal> you can use the following directives
- in Apache's httpd.conf:
- <screen>
- <VirtualHost *>
- ErrorLog /home/srw/logs/error_log
- TransferLog /home/srw/logs/access_log
- ProxyPass / http://srw.mydomain:210/
- </VirtualHost>
- </screen>
- </para>
- <para>
- The above for the Apache 1.3 series.
- </para>
- </example>
- <example id="server.example.local.access">
- <title>Running a server with local access only</title>
- <para>
- Servers that is only being accessed from the local host should listen
- on UNIX file socket rather than a Internet socket. To listen on
- <filename>/tmp/mysocket</filename> start the server as follows:
- <screen>
- application unix:/tmp/mysocket
- </screen>
- </para>
- </example>
- </sect1>
- <sect1 id="server.vhosts"><title>GFS Configuration and Virtual Hosts</title>
- &gfs-virtual;
- </sect1>
- </chapter>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
+++ /dev/null
-<chapter id="soap"><title>SOAP and SRU</title>
- <sect1 id="soap.introduction"><title>Introduction</title>
- <para>
- &yaz; uses a very simple implementation of
- <ulink url="&url.soap;">SOAP</ulink> that only,
- currenly, supports what is sufficient to offer SRU SOAP functionality.
- The implementation uses the
- <ulink url="&url.libxml2.api.tree;">tree API</ulink> of
- libxml2 to encode and decode SOAP packages.
- </para>
- <para>
- Like the Z39.50 ASN.1 module, the &yaz; SRU implementation uses
- simple C structs to represent SOAP packages as well as
- HTTP packages.
- </para>
- </sect1>
- <sect1 id="soap.http"><title>HTTP</title>
- <para>
- &yaz; only offers HTTP as transport carrier for SOAP, but it is
- relatively easy to change that.
- </para>
- <para>
- The following definition of <literal>Z_GDU</literal> (Generic Data
- Unit) allows for both HTTP and Z39.50 in one packet.
- </para>
- <synopsis>
-#include <yaz/zgdu.h>
-
-#define Z_GDU_Z3950 1
-#define Z_GDU_HTTP_Request 2
-#define Z_GDU_HTTP_Response 3
-typedef struct {
- int which;
- union {
- Z_APDU *z3950;
- Z_HTTP_Request *HTTP_Request;
- Z_HTTP_Response *HTTP_Response;
- } u;
-} Z_GDU ;
- </synopsis>
- <para>
- The corresponding Z_GDU encoder/decoder is <function>z_GDU</function>.
- The <literal>z3950</literal> is any of the known BER encoded Z39.50
- APDUs.
- <literal>HTTP_Request</literal> and <literal>HTTP_Response</literal>
- is the HTTP Request and Response respectively.
- </para>
- </sect1>
- <sect1 id="soap.xml"><title>SOAP Packages</title>
- <para>
- Every SOAP package in &yaz; is represented as follows:
- <synopsis>
-#include <yaz/soap.h>
-
-typedef struct {
- char *fault_code;
- char *fault_string;
- char *details;
-} Z_SOAP_Fault;
-
-typedef struct {
- int no;
- char *ns;
- void *p;
-} Z_SOAP_Generic;
-
-#define Z_SOAP_fault 1
-#define Z_SOAP_generic 2
-#define Z_SOAP_error 3
-typedef struct {
- int which;
- union {
- Z_SOAP_Fault *fault;
- Z_SOAP_Generic *generic;
- Z_SOAP_Fault *soap_error;
- } u;
- const char *ns;
-} Z_SOAP;
- </synopsis>
- </para>
- <para>
- The <literal>fault</literal> and <literal>soap_error</literal>
- arms represent both a SOAP fault - struct
- <literal>Z_SOAP_Fault</literal>. Any other generic
- (valid) package is represented by <literal>Z_SOAP_Generic</literal>.
- </para>
- <para>
- The <literal>ns</literal> as part of <literal>Z_SOAP</literal>
- is the namespace for SOAP itself and reflects the SOAP
- version. For version 1.1 it is
- <literal>http://schemas.xmlsoap.org/soap/envelope/</literal>,
- for version 1.2 it is
- <literal>http://www.w3.org/2001/06/soap-envelope</literal>.
- </para>
- <synopsis>
-int z_soap_codec(ODR o, Z_SOAP **pp,
- char **content_buf, int *content_len,
- Z_SOAP_Handler *handlers);
- </synopsis>
- <para>
- The <literal>content_buf</literal> and <literal>content_len</literal>
- is XML buffer and length of buffer respectively.
- </para>
- <para>
- The <literal>handlers</literal> is a list of SOAP codec
- handlers - one handler for each service namespace. For SRU SOAP, the
- namespace would be <literal>http://www.loc.gov/zing/srw/v1.0/</literal>.
- </para>
- <para>
- When decoding, the <function>z_soap_codec</function>
- inspects the XML content
- and tries to match one of the services namespaces of the
- supplied handlers. If there is a match a handler function
- is invoked which decodes that particular SOAP package.
- If successful, the returned <literal>Z_SOAP</literal> package will be
- of type <literal>Z_SOAP_Generic</literal>.
- Member <literal>no</literal> is
- set the offset of handler that matched; <literal>ns</literal>
- is set to namespace of matching handler; the void pointer
- <literal>p</literal> is set to the C data structure assocatiated
- with the handler.
- </para>
- <para>
- When a NULL namespace is met (member <literal>ns</literal> bwlow),
- that specifies end-of-list.
- </para>
- <para>
- Each handler is defined as follows:
- <synopsis>
-typedef struct {
- char *ns;
- void *client_data;
- Z_SOAP_fun f;
-} Z_SOAP_Handler;
- </synopsis>
- The <literal>ns</literal> is namespace of service associated with
- handler <literal>f</literal>. <literal>client_data</literal>
- is user-defined data which is passed to handler.
- </para>
- <para>
- The prototype for a SOAP service handler is:
- <synopsis>
-int handler(ODR o, void * ptr, void **handler_data,
- void *client_data, const char *ns);
- </synopsis>
- The <parameter>o</parameter> specifies the mode (decode/encode)
- as usual. The second argument, <parameter>ptr</parameter>,
- is a libxml2 tree node pointer (<literal>xmlNodePtr</literal>)
- and is a pointer to the <literal>Body</literal> element
- of the SOAP package. The <parameter>handler_data</parameter>
- is an opaque pointer to a C definitions associated with the
- SOAP service. <parameter>client_data</parameter> is the pointer
- which was set as part of the <literal>Z_SOAP_handler</literal>.
- Finally, <parameter>ns</parameter> the service namespace.
- </para>
- </sect1>
- <sect1 id="soap.srw"><title>SRU</title>
- <para>
- SRU SOAP is just one implementation of a SOAP handler as described
- in the previous section.
- The encoder/decoder handler for SRU is defined as
- follows:
- <synopsis>
-#include <yaz/srw.h>
-
-int yaz_srw_codec(ODR o, void * pptr,
- Z_SRW_GDU **handler_data,
- void *client_data, const char *ns);
- </synopsis>
- Here, <literal>Z_SRW_GDU</literal> is either
- searchRetrieveRequest or a searchRetrieveResponse.
- </para>
- <note>
- <para>
- The xQuery and xSortKeys are not handled yet by
- the SRW implementation of &yaz;. Explain is also missing.
- Future versions of &yaz; will include these features.
- </para>
- </note>
- <para>
- The definition of searchRetrieveRequest is:
- <synopsis>
-typedef struct {
-
-#define Z_SRW_query_type_cql 1
-#define Z_SRW_query_type_xcql 2
-#define Z_SRW_query_type_pqf 3
- int query_type;
- union {
- char *cql;
- char *xcql;
- char *pqf;
- } query;
-
-#define Z_SRW_sort_type_none 1
-#define Z_SRW_sort_type_sort 2
-#define Z_SRW_sort_type_xSort 3
- int sort_type;
- union {
- char *none;
- char *sortKeys;
- char *xSortKeys;
- } sort;
- int *startRecord;
- int *maximumRecords;
- char *recordSchema;
- char *recordPacking;
- char *database;
-} Z_SRW_searchRetrieveRequest;
- </synopsis>
- Please observe that data of type xsd:string is represented
- as a char pointer (<literal>char *</literal>). A null pointer
- means that the element is absent.
- Data of type xsd:integer is representd as a pointer to
- an int (<literal>int *</literal>). Again, a null pointer
- us used for absent elements.
- </para>
- <para>
- The SearchRetrieveResponse has the following definition.
- <synopsis>
-typedef struct {
- int * numberOfRecords;
- char * resultSetId;
- int * resultSetIdleTime;
-
- Z_SRW_record *records;
- int num_records;
-
- Z_SRW_diagnostic *diagnostics;
- int num_diagnostics;
- int *nextRecordPosition;
-} Z_SRW_searchRetrieveResponse;
- </synopsis>
- The <literal>num_records</literal> and <literal>num_diagnostics</literal>
- is number of returned records and diagnostics respectively and also
- correspond to the "size of" arrays <literal>records</literal>
- and <literal>diagnostics</literal>.
- </para>
- <para>
- A retrieval record is defined as follows:
- <synopsis>
-typedef struct {
- char *recordSchema;
- char *recordData_buf;
- int recordData_len;
- int *recordPosition;
-} Z_SRW_record;
- </synopsis>
- The record data is defined as a buffer of some length so that
- data can be of any type. SRW 1.0 currenly doesn't allow for this
- (only XML), but future versions might do.
- </para>
- <para>
- And, a diagnostic as:
- <synopsis>
-typedef struct {
- int *code;
- char *details;
-} Z_SRW_diagnostic;
- </synopsis>
- </para>
- </sect1>
-</chapter>
-
+++ /dev/null
- <chapter id="tools"><title>Supporting Tools</title>
-
- <para>
- In support of the service API - primarily the ASN module, which
- provides the pro-grammatic interface to the Z39.50 APDUs, &yaz; contains
- a collection of tools that support the development of applications.
- </para>
-
- <sect1 id="tools.query"><title>Query Syntax Parsers</title>
-
- <para>
- Since the type-1 (RPN) query structure has no direct, useful string
- representation, every origin application needs to provide some form of
- mapping from a local query notation or representation to a
- <token>Z_RPNQuery</token> structure. Some programmers will prefer to
- construct the query manually, perhaps using
- <function>odr_malloc()</function> to simplify memory management.
- The &yaz; distribution includes three separate, query-generating tools
- that may be of use to you.
- </para>
-
- <sect2 id="PQF"><title>Prefix Query Format</title>
-
- <para>
- Since RPN or reverse polish notation is really just a fancy way of
- describing a suffix notation format (operator follows operands), it
- would seem that the confusion is total when we now introduce a prefix
- notation for RPN. The reason is one of simple laziness - it's somewhat
- simpler to interpret a prefix format, and this utility was designed
- for maximum simplicity, to provide a baseline representation for use
- in simple test applications and scripting environments (like Tcl). The
- demonstration client included with YAZ uses the PQF.
- </para>
-
- <note>
- <para>
- The PQF have been adopted by other parties developing Z39.50
- software. It is often referred to as Prefix Query Notation
- - PQN.
- </para>
- </note>
- <para>
- The PQF is defined by the pquery module in the YAZ library.
- There are two sets of function that have similar behavior. First
- set operates on a PQF parser handle, second set doesn't. First set
- set of functions are more flexible than the second set. Second set
- is obsolete and is only provided to ensure backwards compatibility.
- </para>
- <para>
- First set of functions all operate on a PQF parser handle:
- </para>
- <synopsis>
- #include <yaz/pquery.h>
-
- YAZ_PQF_Parser yaz_pqf_create(void);
-
- void yaz_pqf_destroy(YAZ_PQF_Parser p);
-
- Z_RPNQuery *yaz_pqf_parse(YAZ_PQF_Parser p, ODR o, const char *qbuf);
-
- Z_AttributesPlusTerm *yaz_pqf_scan(YAZ_PQF_Parser p, ODR o,
- Odr_oid **attributeSetId, const char *qbuf);
-
- int yaz_pqf_error(YAZ_PQF_Parser p, const char **msg, size_t *off);
- </synopsis>
- <para>
- A PQF parser is created and destructed by functions
- <function>yaz_pqf_create</function> and
- <function>yaz_pqf_destroy</function> respectively.
- Function <function>yaz_pqf_parse</function> parses query given
- by string <literal>qbuf</literal>. If parsing was successful,
- a Z39.50 RPN Query is returned which is created using ODR stream
- <literal>o</literal>. If parsing failed, a NULL pointer is
- returned.
- Function <function>yaz_pqf_scan</function> takes a scan query in
- <literal>qbuf</literal>. If parsing was successful, the function
- returns attributes plus term pointer and modifies
- <literal>attributeSetId</literal> to hold attribute set for the
- scan request - both allocated using ODR stream <literal>o</literal>.
- If parsing failed, yaz_pqf_scan returns a NULL pointer.
- Error information for bad queries can be obtained by a call to
- <function>yaz_pqf_error</function> which returns an error code and
- modifies <literal>*msg</literal> to point to an error description,
- and modifies <literal>*off</literal> to the offset within last
- query were parsing failed.
- </para>
- <para>
- The second set of functions are declared as follows:
- </para>
- <synopsis>
- #include <yaz/pquery.h>
-
- Z_RPNQuery *p_query_rpn(ODR o, oid_proto proto, const char *qbuf);
-
- Z_AttributesPlusTerm *p_query_scan(ODR o, oid_proto proto,
- Odr_oid **attributeSetP, const char *qbuf);
-
- int p_query_attset(const char *arg);
- </synopsis>
- <para>
- The function <function>p_query_rpn()</function> takes as arguments an
- &odr; stream (see section <link linkend="odr">The ODR Module</link>)
- to provide a memory source (the structure created is released on
- the next call to <function>odr_reset()</function> on the stream), a
- protocol identifier (one of the constants <token>PROTO_Z3950</token> and
- <token>PROTO_SR</token>), an attribute set reference, and
- finally a null-terminated string holding the query string.
- </para>
- <para>
- If the parse went well, <function>p_query_rpn()</function> returns a
- pointer to a <literal>Z_RPNQuery</literal> structure which can be
- placed directly into a <literal>Z_SearchRequest</literal>.
- If parsing failed, due to syntax error, a NULL pointer is returned.
- </para>
- <para>
- The <literal>p_query_attset</literal> specifies which attribute set
- to use if the query doesn't specify one by the
- <literal>@attrset</literal> operator.
- The <literal>p_query_attset</literal> returns 0 if the argument is a
- valid attribute set specifier; otherwise the function returns -1.
- </para>
-
- <para>
- The grammar of the PQF is as follows:
- </para>
-
- <literallayout>
- query ::= top-set query-struct.
-
- top-set ::= [ '@attrset' string ]
-
- query-struct ::= attr-spec | simple | complex | '@term' term-type query
-
- attr-spec ::= '@attr' [ string ] string query-struct
-
- complex ::= operator query-struct query-struct.
-
- operator ::= '@and' | '@or' | '@not' | '@prox' proximity.
-
- simple ::= result-set | term.
-
- result-set ::= '@set' string.
-
- term ::= string.
-
- proximity ::= exclusion distance ordered relation which-code unit-code.
-
- exclusion ::= '1' | '0' | 'void'.
-
- distance ::= integer.
-
- ordered ::= '1' | '0'.
-
- relation ::= integer.
-
- which-code ::= 'known' | 'private' | integer.
-
- unit-code ::= integer.
-
- term-type ::= 'general' | 'numeric' | 'string' | 'oid' | 'datetime' | 'null'.
- </literallayout>
-
- <para>
- You will note that the syntax above is a fairly faithful
- representation of RPN, except for the Attribute, which has been
- moved a step away from the term, allowing you to associate one or more
- attributes with an entire query structure. The parser will
- automatically apply the given attributes to each term as required.
- </para>
-
- <para>
- The @attr operator is followed by an attribute specification
- (<literal>attr-spec</literal> above). The specification consists
- of an optional attribute set, an attribute type-value pair and
- a sub-query. The attribute type-value pair is packed in one string:
- an attribute type, an equals sign, and an attribute value, like this:
- <literal>@attr 1=1003</literal>.
- The type is always an integer but the value may be either an
- integer or a string (if it doesn't start with a digit character).
- A string attribute-value is encoded as a Type-1 ``complex''
- attribute with the list of values containing the single string
- specified, and including no semantic indicators.
- </para>
-
- <para>
- Version 3 of the Z39.50 specification defines various encoding of terms.
- Use <literal>@term </literal> <replaceable>type</replaceable>
- <replaceable>string</replaceable>,
- where type is one of: <literal>general</literal>,
- <literal>numeric</literal> or <literal>string</literal>
- (for InternationalString).
- If no term type has been given, the <literal>general</literal> form
- is used. This is the only encoding allowed in both versions 2 and 3
- of the Z39.50 standard.
- </para>
-
- <sect3 id="PQF-prox">
- <title>Using Proximity Operators with PQF</title>
- <note>
- <para>
- This is an advanced topic, describing how to construct
- queries that make very specific requirements on the
- relative location of their operands.
- You may wish to skip this section and go straight to
- <link linkend="pqf-examples">the example PQF queries</link>.
- </para>
- <para>
- <warning>
- <para>
- Most Z39.50 servers do not support proximity searching, or
- support only a small subset of the full functionality that
- can be expressed using the PQF proximity operator. Be
- aware that the ability to <emphasis>express</emphasis> a
- query in PQF is no guarantee that any given server will
- be able to <emphasis>execute</emphasis> it.
- </para>
- </warning>
- </para>
- </note>
- <para>
- The proximity operator <literal>@prox</literal> is a special
- and more restrictive version of the conjunction operator
- <literal>@and</literal>. Its semantics are described in
- section 3.7.2 (Proximity) of Z39.50 the standard itself, which
- can be read on-line at
- <ulink url="&url.z39.50.proximity;"/>
- </para>
- <para>
- In PQF, the proximity operation is represented by a sequence
- of the form
- <screen>
-@prox <replaceable>exclusion</replaceable> <replaceable>distance</replaceable> <replaceable>ordered</replaceable> <replaceable>relation</replaceable> <replaceable>which-code</replaceable> <replaceable>unit-code</replaceable>
- </screen>
- in which the meanings of the parameters are as described in in
- the standard, and they can take the following values:
- <itemizedlist>
- <listitem><formalpara><title>exclusion</title><para>
- 0 = false (i.e. the proximity condition specified by the
- remaining parameters must be satisfied) or
- 1 = true (the proximity condition specified by the
- remaining parameters must <emphasis>not</emphasis> be
- satisifed).
- </para></formalpara></listitem>
- <listitem><formalpara><title>distance</title><para>
- An integer specifying the difference between the locations
- of the operands: e.g. two adjacent words would have
- distance=1 since their locations differ by one unit.
- </para></formalpara></listitem>
- <listitem><formalpara><title>ordered</title><para>
- 1 = ordered (the operands must occur in the order the
- query specifies them) or
- 0 = unordered (they may appear in either order).
- </para></formalpara></listitem>
- <listitem><formalpara><title>relation</title><para>
- Recognised values are
- 1 (lessThan),
- 2 (lessThanOrEqual),
- 3 (equal),
- 4 (greaterThanOrEqual),
- 5 (greaterThan) and
- 6 (notEqual).
- </para></formalpara></listitem>
- <listitem><formalpara><title>which-code</title><para>
- <literal>known</literal>
- or
- <literal>k</literal>
- (the unit-code parameter is taken from the well-known list
- of alternatives described in below) or
- <literal>private</literal>
- or
- <literal>p</literal>
- (the unit-code paramater has semantics specific to an
- out-of-band agreement such as a profile).
- </para></formalpara></listitem>
- <listitem><formalpara><title>unit-code</title><para>
- If the which-code parameter is <literal>known</literal>
- then the recognised values are
- 1 (character),
- 2 (word),
- 3 (sentence),
- 4 (paragraph),
- 5 (section),
- 6 (chapter),
- 7 (document),
- 8 (element),
- 9 (subelement),
- 10 (elementType) and
- 11 (byte).
- If which-code is <literal>private</literal> then the
- acceptable values are determined by the profile.
- </para></formalpara></listitem>
- </itemizedlist>
- (The numeric values of the relation and well-known unit-code
- parameters are taken straight from
- <ulink url="&url.z39.50.proximity.asn1;"
- >the ASN.1</ulink> of the proximity structure in the standard.)
- </para>
- </sect3>
-
- <sect3 id="pqf-examples"><title>PQF queries</title>
-
- <example id="example.pqf.simple.terms">
- <title>PQF queries using simple terms</title>
- <para>
- <screen>
- dylan
-
- "bob dylan"
- </screen>
- </para>
- </example>
- <example id="pqf.example.pqf.boolean.operators">
- <title>PQF boolean operators</title>
- <para>
- <screen>
- @or "dylan" "zimmerman"
-
- @and @or dylan zimmerman when
-
- @and when @or dylan zimmerman
- </screen>
- </para>
- </example>
- <example id="example.pqf.result.sets">
- <title>PQF references to result sets</title>
- <para>
- <screen>
- @set Result-1
-
- @and @set seta @set setb
- </screen>
- </para>
- </example>
- <example id="example.pqf.attributes">
- <title>Attributes for terms</title>
- <para>
- <screen>
- @attr 1=4 computer
-
- @attr 1=4 @attr 4=1 "self portrait"
-
- @attrset exp1 @attr 1=1 CategoryList
-
- @attr gils 1=2008 Copenhagen
-
- @attr 1=/book/title computer
- </screen>
- </para>
- </example>
- <example id="example.pqf.proximity">
- <title>PQF Proximity queries</title>
- <para>
- <screen>
- @prox 0 3 1 2 k 2 dylan zimmerman
- </screen>
- <note><para>
- Here the parameters 0, 3, 1, 2, k and 2 represent exclusion,
- distance, ordered, relation, which-code and unit-code, in that
- order. So:
- <itemizedlist>
- <listitem><para>
- exclusion = 0: the proximity condition must hold
- </para></listitem>
- <listitem><para>
- distance = 3: the terms must be three units apart
- </para></listitem>
- <listitem><para>
- ordered = 1: they must occur in the order they are specified
- </para></listitem>
- <listitem><para>
- relation = 2: lessThanOrEqual (to the distance of 3 units)
- </para></listitem>
- <listitem><para>
- which-code is ``known'', so the standard unit-codes are used
- </para></listitem>
- <listitem><para>
- unit-code = 2: word.
- </para></listitem>
- </itemizedlist>
- So the whole proximity query means that the words
- <literal>dylan</literal> and <literal>zimmerman</literal> must
- both occur in the record, in that order, differing in position
- by three or fewer words (i.e. with two or fewer words between
- them.) The query would find ``Bob Dylan, aka. Robert
- Zimmerman'', but not ``Bob Dylan, born as Robert Zimmerman''
- since the distance in this case is four.
- </para></note>
- </para>
- </example>
- <example id="example.pqf.search.term.type">
- <title>PQF specification of search term type</title>
- <para>
- <screen>
- @term string "a UTF-8 string, maybe?"
- </screen>
- </para>
- </example>
- <example id="example.pqf.mixed.queries">
- <title>PQF mixed queries</title>
- <para>
- <screen>
- @or @and bob dylan @set Result-1
-
- @attr 4=1 @and @attr 1=1 "bob dylan" @attr 1=4 "slow train coming"
-
- @and @attr 2=4 @attr gils 1=2038 -114 @attr 2=2 @attr gils 1=2039 -109
- </screen>
- <note>
- <para>
- The last of these examples is a spatial search: in
- <ulink url="http://www.gils.net/prof_v2.html#sec_7_4"
- >the GILS attribute set</ulink>,
- access point
- 2038 indicates West Bounding Coordinate and
- 2030 indicates East Bounding Coordinate,
- so the query is for areas extending from -114 degrees
- to no more than -109 degrees.
- </para>
- </note>
- </para>
- </example>
- </sect3>
- </sect2>
- <sect2 id="CCL"><title>CCL</title>
-
- <para>
- Not all users enjoy typing in prefix query structures and numerical
- attribute values, even in a minimalistic test client. In the library
- world, the more intuitive Common Command Language - CCL (ISO 8777)
- has enjoyed some popularity - especially before the widespread
- availability of graphical interfaces. It is still useful in
- applications where you for some reason or other need to provide a
- symbolic language for expressing boolean query structures.
- </para>
-
- <sect3 id="ccl.syntax">
- <title>CCL Syntax</title>
-
- <para>
- The CCL parser obeys the following grammar for the FIND argument.
- The syntax is annotated by in the lines prefixed by
- <literal>--</literal>.
- </para>
-
- <screen>
- CCL-Find ::= CCL-Find Op Elements
- | Elements.
-
- Op ::= "and" | "or" | "not"
- -- The above means that Elements are separated by boolean operators.
-
- Elements ::= '(' CCL-Find ')'
- | Set
- | Terms
- | Qualifiers Relation Terms
- | Qualifiers Relation '(' CCL-Find ')'
- | Qualifiers '=' string '-' string
- -- Elements is either a recursive definition, a result set reference, a
- -- list of terms, qualifiers followed by terms, qualifiers followed
- -- by a recursive definition or qualifiers in a range (lower - upper).
-
- Set ::= 'set' = string
- -- Reference to a result set
-
- Terms ::= Terms Prox Term
- | Term
- -- Proximity of terms.
-
- Term ::= Term string
- | string
- -- This basically means that a term may include a blank
-
- Qualifiers ::= Qualifiers ',' string
- | string
- -- Qualifiers is a list of strings separated by comma
-
- Relation ::= '=' | '>=' | '<=' | '<>' | '>' | '<'
- -- Relational operators. This really doesn't follow the ISO8777
- -- standard.
-
- Prox ::= '%' | '!'
- -- Proximity operator
-
- </screen>
-
- <example id="example.ccl.queries">
- <title>CCL queries</title>
- <para>
- The following queries are all valid:
- </para>
-
- <screen>
- dylan
-
- "bob dylan"
-
- dylan or zimmerman
-
- set=1
-
- (dylan and bob) or set=1
-
- righttrunc?
-
- "notrunc?"
-
- singlechar#mask
-
- </screen>
- <para>
- Assuming that the qualifiers <literal>ti</literal>,
- <literal>au</literal>
- and <literal>date</literal> are defined we may use:
- </para>
-
- <screen>
- ti=self portrait
-
- au=(bob dylan and slow train coming)
-
- date>1980 and (ti=((self portrait)))
-
- </screen>
- </example>
-
- </sect3>
- <sect3 id="ccl.qualifiers">
- <title>CCL Qualifiers</title>
-
- <para>
- Qualifiers are used to direct the search to a particular searchable
- index, such as title (ti) and author indexes (au). The CCL standard
- itself doesn't specify a particular set of qualifiers, but it does
- suggest a few short-hand notations. You can customize the CCL parser
- to support a particular set of qualifiers to reflect the current target
- profile. Traditionally, a qualifier would map to a particular
- use-attribute within the BIB-1 attribute set. It is also
- possible to set other attributes, such as the structure
- attribute.
- </para>
-
- <para>
- A CCL profile is a set of predefined CCL qualifiers that may be
- read from a file or set in the CCL API.
- The YAZ client reads its CCL qualifiers from a file named
- <filename>default.bib</filename>. There are four types of
- lines in a CCL profile: qualifier specification,
- qualifier alias, comments and directives.
- </para>
- <sect4 id="ccl.qualifier.specification">
- <title>Qualifier specification</title>
- <para>
- A qualifier specification is of the form:
- </para>
-
- <para>
- <replaceable>qualifier-name</replaceable>
- [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable>
- [<replaceable>attributeset</replaceable><literal>,</literal>]<replaceable>type</replaceable><literal>=</literal><replaceable>val</replaceable> ...
- </para>
-
- <para>
- where <replaceable>qualifier-name</replaceable> is the name of the
- qualifier to be used (eg. <literal>ti</literal>),
- <replaceable>type</replaceable> is attribute type in the attribute
- set (Bib-1 is used if no attribute set is given) and
- <replaceable>val</replaceable> is attribute value.
- The <replaceable>type</replaceable> can be specified as an
- integer or as it be specified either as a single-letter:
- <literal>u</literal> for use,
- <literal>r</literal> for relation,<literal>p</literal> for position,
- <literal>s</literal> for structure,<literal>t</literal> for truncation
- or <literal>c</literal> for completeness.
- The attributes for the special qualifier name <literal>term</literal>
- are used when no CCL qualifier is given in a query.
- <table id="ccl.common.bib1.attributes">
- <title>Common Bib-1 attributes</title>
- <tgroup cols="2">
- <colspec colwidth="2*" colname="type"></colspec>
- <colspec colwidth="9*" colname="description"></colspec>
- <thead>
- <row>
- <entry>Type</entry>
- <entry>Description</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry><literal>u=</literal><replaceable>value</replaceable></entry>
- <entry>
- Use attribute (1). Common use attributes are
- 1 Personal-name, 4 Title, 7 ISBN, 8 ISSN, 30 Date,
- 62 Subject, 1003 Author), 1016 Any. Specify value
- as an integer.
- </entry>
- </row>
-
- <row>
- <entry><literal>r=</literal><replaceable>value</replaceable></entry>
- <entry>
- Relation attribute (2). Common values are
- 1 <, 2 <=, 3 =, 4 >=, 5 >, 6 <>,
- 100 phonetic, 101 stem, 102 relevance, 103 always matches.
- </entry>
- </row>
-
- <row>
- <entry><literal>p=</literal><replaceable>value</replaceable></entry>
- <entry>
- Position attribute (3). Values: 1 first in field, 2
- first in any subfield, 3 any position in field.
- </entry>
- </row>
-
- <row>
- <entry><literal>s=</literal><replaceable>value</replaceable></entry>
- <entry>
- Structure attribute (4). Values: 1 phrase, 2 word,
- 3 key, 4 year, 5 date, 6 word list, 100 date (un),
- 101 name (norm), 102 name (un), 103 structure, 104 urx,
- 105 free-form-text, 106 document-text, 107 local-number,
- 108 string, 109 numeric string.
- </entry>
- </row>
-
- <row>
- <entry><literal>t=</literal><replaceable>value</replaceable></entry>
- <entry>
- Truncation attribute (5). Values: 1 right, 2 left,
- 3 left& right, 100 none, 101 process #, 102 regular-1,
- 103 regular-2, 104 CCL.
- </entry>
- </row>
-
- <row>
- <entry><literal>c=</literal><replaceable>value</replaceable></entry>
- <entry>
- Completeness attribute (6). Values: 1 incomplete subfield,
- 2 complete subfield, 3 complete field.
- </entry>
- </row>
-
- </tbody>
- </tgroup>
- </table>
- </para>
- <para>
- Refer to <xref linkend="bib1"/> or the complete
- <ulink url="&url.z39.50.attset.bib1;">list of Bib-1 attributes</ulink>
- </para>
- <para>
- It is also possible to specify non-numeric attribute values,
- which are used in combination with certain types.
- The special combinations are:
-
- <table id="ccl.special.attribute.combos">
- <title>Special attribute combos</title>
- <tgroup cols="2">
- <colspec colwidth="2*" colname="name"></colspec>
- <colspec colwidth="9*" colname="description"></colspec>
- <thead>
- <row>
- <entry>Name</entry>
- <entry>Description</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry><literal>s=pw</literal></entry><entry>
- The structure is set to either word or phrase depending
- on the number of tokens in a term (phrase-word).
- </entry>
- </row>
- <row>
- <entry><literal>s=al</literal></entry><entry>
- Each token in the term is ANDed. (and-list).
- This does not set the structure at all.
- </entry>
- </row>
-
- <row><entry><literal>s=ol</literal></entry><entry>
- Each token in the term is ORed. (or-list).
- This does not set the structure at all.
- </entry>
- </row>
-
- <row><entry><literal>s=ag</literal></entry><entry>
- Tokens that appears as phrases (with blank in them) gets
- structure phrase attached (4=1). Tokens that appear to be words
- gets structure word attached (4=2). Phrases and words are
- ANDed. This is a variant of s=al and s=pw, with the main
- difference that words are not split (with operator AND)
- but instead kept in one RPN token. This facility appeared
- in YAZ 4.2.38.
- </entry>
- </row>
-
- <row><entry><literal>r=o</literal></entry><entry>
- Allows ranges and the operators greather-than, less-than, ...
- equals.
- This sets Bib-1 relation attribute accordingly (relation
- ordered). A query construct is only treated as a range if
- dash is used and that is surrounded by white-space. So
- <literal>-1980</literal> is treated as term
- <literal>"-1980"</literal> not <literal><= 1980</literal>.
- If <literal>- 1980</literal> is used, however, that is
- treated as a range.
- </entry>
- </row>
-
- <row><entry><literal>r=r</literal></entry><entry>
- Similar to <literal>r=o</literal> but assumes that terms
- are non-negative (not prefixed with <literal>-</literal>).
- Thus, a dash will always be treated as a range.
- The construct <literal>1980-1990</literal> is
- treated as a range with <literal>r=r</literal> but as a
- single term <literal>"1980-1990"</literal> with
- <literal>r=o</literal>. The special attribute
- <literal>r=r</literal> is available in YAZ 2.0.24 or later.
- </entry>
- </row>
-
- <row><entry><literal>t=l</literal></entry><entry>
- Allows term to be left-truncated.
- If term is of the form <literal>?x</literal>, the resulting
- Type-1 term is <literal>x</literal> and truncation is left.
- </entry>
- </row>
-
- <row><entry><literal>t=r</literal></entry><entry>
- Allows term to be right-truncated.
- If term is of the form <literal>x?</literal>, the resulting
- Type-1 term is <literal>x</literal> and truncation is right.
- </entry>
- </row>
-
- <row><entry><literal>t=n</literal></entry><entry>
- If term is does not include <literal>?</literal>, the
- truncation attribute is set to none (100).
- </entry>
- </row>
-
- <row><entry><literal>t=b</literal></entry><entry>
- Allows term to be both left&right truncated.
- If term is of the form <literal>?x?</literal>, the
- resulting term is <literal>x</literal> and trunctation is
- set to both left&right.
- </entry>
- </row>
-
- <row><entry><literal>t=x</literal></entry><entry>
- Allows masking anywhere in a term, thus fully supporting
- # (mask one character) and ? (zero or more of any).
- If masking is used, trunction is set to 102 (regexp-1 in term)
- and the term is converted accordingly to a regular expression.
- </entry>
- </row>
-
- <row><entry><literal>t=z</literal></entry><entry>
- Allows masking anywhere in a term, thus fully supporting
- # (mask one character) and ? (zero or more of any).
- If masking is used, trunction is set to 104 (Z39.58 in term)
- and the term is converted accordingly to Z39.58 masking term -
- actually the same truncation as CCL itself.
- </entry>
- </row>
-
- </tbody>
- </tgroup>
- </table>
- </para>
- <example id="example.ccl.profile"><title>CCL profile</title>
- <para>
- Consider the following definition:
- </para>
-
- <screen>
- ti u=4 s=1
- au u=1 s=1
- term s=105
- ranked r=102
- date u=30 r=o
- </screen>
- <para>
- <literal>ti</literal> and <literal>au</literal> both set
- structure attribute to phrase (s=1).
- <literal>ti</literal>
- sets the use-attribute to 4. <literal>au</literal> sets the
- use-attribute to 1.
- When no qualifiers are used in the query the structure-attribute is
- set to free-form-text (105) (rule for <literal>term</literal>).
- The <literal>date</literal> sets the relation attribute to
- the relation used in the CCL query and sets the use attribute
- to 30 (Bib-1 Date).
- </para>
- <para>
- You can combine attributes. To Search for "ranked title" you
- can do
- <screen>
- ti,ranked=knuth computer
- </screen>
- which will set relation=ranked, use=title, structure=phrase.
- </para>
- <para>
- Query
- <screen>
- date > 1980
- </screen>
- is a valid query. But
- <screen>
- ti > 1980
- </screen>
- is invalid.
- </para>
- </example>
- </sect4>
- <sect4 id="ccl.qualifier.alias">
- <title>Qualifier alias</title>
- <para>
- A qualifier alias is of the form:
- </para>
- <para>
- <replaceable>q</replaceable>
- <replaceable>q1</replaceable> <replaceable>q2</replaceable> ..
- </para>
- <para>
- which declares <replaceable>q</replaceable> to
- be an alias for <replaceable>q1</replaceable>,
- <replaceable>q2</replaceable>... such that the CCL
- query <replaceable>q=x</replaceable> is equivalent to
- <replaceable>q1=x or q2=x or ...</replaceable>.
- </para>
- </sect4>
-
- <sect4 id="ccl.comments">
- <title>Comments</title>
- <para>
- Lines with white space or lines that begin with
- character <literal>#</literal> are treated as comments.
- </para>
- </sect4>
-
- <sect4 id="ccl.directives">
- <title>Directives</title>
- <para>
- Directive specifications takes the form
- </para>
- <para><literal>@</literal><replaceable>directive</replaceable> <replaceable>value</replaceable>
- </para>
- <table id="ccl.directives.table">
- <title>CCL directives</title>
- <tgroup cols="3">
- <colspec colwidth="2*" colname="name"></colspec>
- <colspec colwidth="8*" colname="description"></colspec>
- <colspec colwidth="1*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Name</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>truncation</entry>
- <entry>Truncation character</entry>
- <entry><literal>?</literal></entry>
- </row>
- <row>
- <entry>mask</entry>
- <entry>Masking character. Requires YAZ 4.2.58 or later</entry>
- <entry><literal>#</literal></entry>
- </row>
- <row>
- <entry>field</entry>
- <entry>Specifies how multiple fields are to be
- combined. There are two modes: <literal>or</literal>:
- multiple qualifier fields are ORed,
- <literal>merge</literal>: attributes for the qualifier
- fields are merged and assigned to one term.
- </entry>
- <entry><literal>merge</literal></entry>
- </row>
- <row>
- <entry>case</entry>
- <entry>Specifies if CCL operators and qualifiers should be
- compared with case sensitivity or not. Specify 1 for
- case sensitive; 0 for case insensitive.</entry>
- <entry><literal>1</literal></entry>
- </row>
-
- <row>
- <entry>and</entry>
- <entry>Specifies token for CCL operator AND.</entry>
- <entry><literal>and</literal></entry>
- </row>
-
- <row>
- <entry>or</entry>
- <entry>Specifies token for CCL operator OR.</entry>
- <entry><literal>or</literal></entry>
- </row>
-
- <row>
- <entry>not</entry>
- <entry>Specifies token for CCL operator NOT.</entry>
- <entry><literal>not</literal></entry>
- </row>
-
- <row>
- <entry>set</entry>
- <entry>Specifies token for CCL operator SET.</entry>
- <entry><literal>set</literal></entry>
- </row>
- </tbody>
- </tgroup>
- </table>
- </sect4>
- </sect3>
- <sect3 id="ccl.api">
- <title>CCL API</title>
- <para>
- All public definitions can be found in the header file
- <filename>ccl.h</filename>. A profile identifier is of type
- <literal>CCL_bibset</literal>. A profile must be created with the call
- to the function <function>ccl_qual_mk</function> which returns a profile
- handle of type <literal>CCL_bibset</literal>.
- </para>
-
- <para>
- To read a file containing qualifier definitions the function
- <function>ccl_qual_file</function> may be convenient. This function
- takes an already opened <literal>FILE</literal> handle pointer as
- argument along with a <literal>CCL_bibset</literal> handle.
- </para>
-
- <para>
- To parse a simple string with a FIND query use the function
- </para>
- <screen>
-struct ccl_rpn_node *ccl_find_str(CCL_bibset bibset, const char *str,
- int *error, int *pos);
- </screen>
- <para>
- which takes the CCL profile (<literal>bibset</literal>) and query
- (<literal>str</literal>) as input. Upon successful completion the RPN
- tree is returned. If an error occur, such as a syntax error, the integer
- pointed to by <literal>error</literal> holds the error code and
- <literal>pos</literal> holds the offset inside query string in which
- the parsing failed.
- </para>
-
- <para>
- An English representation of the error may be obtained by calling
- the <literal>ccl_err_msg</literal> function. The error codes are
- listed in <filename>ccl.h</filename>.
- </para>
-
- <para>
- To convert the CCL RPN tree (type
- <literal>struct ccl_rpn_node *</literal>)
- to the Z_RPNQuery of YAZ the function <function>ccl_rpn_query</function>
- must be used. This function which is part of YAZ is implemented in
- <filename>yaz-ccl.c</filename>.
- After calling this function the CCL RPN tree is probably no longer
- needed. The <literal>ccl_rpn_delete</literal> destroys the CCL RPN tree.
- </para>
-
- <para>
- A CCL profile may be destroyed by calling the
- <function>ccl_qual_rm</function> function.
- </para>
-
- <para>
- The token names for the CCL operators may be changed by setting the
- globals (all type <literal>char *</literal>)
- <literal>ccl_token_and</literal>, <literal>ccl_token_or</literal>,
- <literal>ccl_token_not</literal> and <literal>ccl_token_set</literal>.
- An operator may have aliases, i.e. there may be more than one name for
- the operator. To do this, separate each alias with a space character.
- </para>
- </sect3>
- </sect2>
- <sect2 id="cql"><title>CQL</title>
- <para>
- <ulink url="&url.cql;">CQL</ulink>
- - Common Query Language - was defined for the
- <ulink url="&url.sru;">SRU</ulink> protocol.
- In many ways CQL has a similar syntax to CCL.
- The objective of CQL is different. Where CCL aims to be
- an end-user language, CQL is <emphasis>the</emphasis> protocol
- query language for SRU.
- </para>
- <tip>
- <para>
- If you are new to CQL, read the
- <ulink url="&url.cql.intro;">Gentle Introduction</ulink>.
- </para>
- </tip>
- <para>
- The CQL parser in &yaz; provides the following:
- <itemizedlist>
- <listitem>
- <para>
- It parses and validates a CQL query.
- </para>
- </listitem>
- <listitem>
- <para>
- It generates a C structure that allows you to convert
- a CQL query to some other query language, such as SQL.
- </para>
- </listitem>
- <listitem>
- <para>
- The parser converts a valid CQL query to PQF, thus providing a
- way to use CQL for both SRU servers and Z39.50 targets at the
- same time.
- </para>
- </listitem>
- <listitem>
- <para>
- The parser converts CQL to XCQL.
- XCQL is an XML representation of CQL.
- XCQL is part of the SRU specification. However, since SRU
- supports CQL only, we don't expect XCQL to be widely used.
- Furthermore, CQL has the advantage over XCQL that it is
- easy to read.
- </para>
- </listitem>
- </itemizedlist>
- </para>
- <sect3 id="cql.parsing"><title>CQL parsing</title>
- <para>
- A CQL parser is represented by the <literal>CQL_parser</literal>
- handle. Its contents should be considered &yaz; internal (private).
- <synopsis>
-#include <yaz/cql.h>
-
-typedef struct cql_parser *CQL_parser;
-
-CQL_parser cql_parser_create(void);
-void cql_parser_destroy(CQL_parser cp);
- </synopsis>
- A parser is created by <function>cql_parser_create</function> and
- is destroyed by <function>cql_parser_destroy</function>.
- </para>
- <para>
- To parse a CQL query string, the following function
- is provided:
- <synopsis>
-int cql_parser_string(CQL_parser cp, const char *str);
- </synopsis>
- A CQL query is parsed by the <function>cql_parser_string</function>
- which takes a query <parameter>str</parameter>.
- If the query was valid (no syntax errors), then zero is returned;
- otherwise -1 is returned to indicate a syntax error.
- </para>
- <para>
- <synopsis>
-int cql_parser_stream(CQL_parser cp,
- int (*getbyte)(void *client_data),
- void (*ungetbyte)(int b, void *client_data),
- void *client_data);
-
-int cql_parser_stdio(CQL_parser cp, FILE *f);
- </synopsis>
- The functions <function>cql_parser_stream</function> and
- <function>cql_parser_stdio</function> parses a CQL query
- - just like <function>cql_parser_string</function>.
- The only difference is that the CQL query can be
- fed to the parser in different ways.
- The <function>cql_parser_stream</function> uses a generic
- byte stream as input. The <function>cql_parser_stdio</function>
- uses a <literal>FILE</literal> handle which is opened for reading.
- </para>
- </sect3>
-
- <sect3 id="cql.tree"><title>CQL tree</title>
- <para>
- The the query string is valid, the CQL parser
- generates a tree representing the structure of the
- CQL query.
- </para>
- <para>
- <synopsis>
-struct cql_node *cql_parser_result(CQL_parser cp);
- </synopsis>
- <function>cql_parser_result</function> returns the
- a pointer to the root node of the resulting tree.
- </para>
- <para>
- Each node in a CQL tree is represented by a
- <literal>struct cql_node</literal>.
- It is defined as follows:
- <synopsis>
-#define CQL_NODE_ST 1
-#define CQL_NODE_BOOL 2
-#define CQL_NODE_SORT 3
-struct cql_node {
- int which;
- union {
- struct {
- char *index;
- char *index_uri;
- char *term;
- char *relation;
- char *relation_uri;
- struct cql_node *modifiers;
- } st;
- struct {
- char *value;
- struct cql_node *left;
- struct cql_node *right;
- struct cql_node *modifiers;
- } boolean;
- struct {
- char *index;
- struct cql_node *next;
- struct cql_node *modifiers;
- struct cql_node *search;
- } sort;
- } u;
-};
- </synopsis>
- There are three node types: search term (ST), boolean (BOOL)
- and sortby (SORT).
- A modifier is treated as a search term too.
- </para>
- <para>
- The search term node has five members:
- <itemizedlist>
- <listitem>
- <para>
- <literal>index</literal>: index for search term.
- If an index is unspecified for a search term,
- <literal>index</literal> will be NULL.
- </para>
- </listitem>
- <listitem>
- <para>
- <literal>index_uri</literal>: index URi for search term
- or NULL if none could be resolved for the index.
- </para>
- </listitem>
- <listitem>
- <para>
- <literal>term</literal>: the search term itself.
- </para>
- </listitem>
- <listitem>
- <para>
- <literal>relation</literal>: relation for search term.
- </para>
- </listitem>
- <listitem>
- <para>
- <literal>relation_uri</literal>: relation URI for search term.
- </para>
- </listitem>
- <listitem>
- <para>
- <literal>modifiers</literal>: relation modifiers for search
- term. The <literal>modifiers</literal> list itself of cql_nodes
- each of type <literal>ST</literal>.
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>
- The boolean node represents <literal>and</literal>,
- <literal>or</literal>, <literal>not</literal> +
- proximity.
- <itemizedlist>
- <listitem>
- <para>
- <literal>left</literal> and <literal>right</literal>: left
- - and right operand respectively.
- </para>
- </listitem>
- <listitem>
- <para>
- <literal>modifiers</literal>: proximity arguments.
- </para>
- </listitem>
- </itemizedlist>
- </para>
-
- <para>
- The sort node represents both the SORTBY clause.
- </para>
-
- </sect3>
- <sect3 id="cql.to.pqf"><title>CQL to PQF conversion</title>
- <para>
- Conversion to PQF (and Z39.50 RPN) is tricky by the fact
- that the resulting RPN depends on the Z39.50 target
- capabilities (combinations of supported attributes).
- In addition, the CQL and SRU operates on index prefixes
- (URI or strings), whereas the RPN uses Object Identifiers
- for attribute sets.
- </para>
- <para>
- The CQL library of &yaz; defines a <literal>cql_transform_t</literal>
- type. It represents a particular mapping between CQL and RPN.
- This handle is created and destroyed by the functions:
- <synopsis>
-cql_transform_t cql_transform_open_FILE (FILE *f);
-cql_transform_t cql_transform_open_fname(const char *fname);
-void cql_transform_close(cql_transform_t ct);
- </synopsis>
- The first two functions create a tranformation handle from
- either an already open FILE or from a filename respectively.
- </para>
- <para>
- The handle is destroyed by <function>cql_transform_close</function>
- in which case no further reference of the handle is allowed.
- </para>
- <para>
- When a <literal>cql_transform_t</literal> handle has been created
- you can convert to RPN.
- <synopsis>
-int cql_transform_buf(cql_transform_t ct,
- struct cql_node *cn, char *out, int max);
- </synopsis>
- This function converts the CQL tree <literal>cn</literal>
- using handle <literal>ct</literal>.
- For the resulting PQF, you supply a buffer <literal>out</literal>
- which must be able to hold at at least <literal>max</literal>
- characters.
- </para>
- <para>
- If conversion failed, <function>cql_transform_buf</function>
- returns a non-zero SRU error code; otherwise zero is returned
- (conversion successful). The meanings of the numeric error
- codes are listed in the SRU specification somewhere (no
- direct link anymore).
- </para>
- <para>
- If conversion fails, more information can be obtained by calling
- <synopsis>
-int cql_transform_error(cql_transform_t ct, char **addinfop);
- </synopsis>
- This function returns the most recently returned numeric
- error-code and sets the string-pointer at
- <literal>*addinfop</literal> to point to a string containing
- additional information about the error that occurred: for
- example, if the error code is 15 (``Illegal or unsupported context
- set''), the additional information is the name of the requested
- context set that was not recognised.
- </para>
- <para>
- The SRU error-codes may be translated into brief human-readable
- error messages using
- <synopsis>
-const char *cql_strerror(int code);
- </synopsis>
- </para>
- <para>
- If you wish to be able to produce a PQF result in a different
- way, there are two alternatives.
- <synopsis>
-void cql_transform_pr(cql_transform_t ct,
- struct cql_node *cn,
- void (*pr)(const char *buf, void *client_data),
- void *client_data);
-
-int cql_transform_FILE(cql_transform_t ct,
- struct cql_node *cn, FILE *f);
- </synopsis>
- The former function produces output to a user-defined
- output stream. The latter writes the result to an already
- open <literal>FILE</literal>.
- </para>
- </sect3>
- <sect3 id="cql.to.rpn">
- <title>Specification of CQL to RPN mappings</title>
- <para>
- The file supplied to functions
- <function>cql_transform_open_FILE</function>,
- <function>cql_transform_open_fname</function> follows
- a structure found in many Unix utilities.
- It consists of mapping specifications - one per line.
- Lines starting with <literal>#</literal> are ignored (comments).
- </para>
- <para>
- Each line is of the form
- <literallayout>
- <replaceable>CQL pattern</replaceable><literal> = </literal> <replaceable> RPN equivalent</replaceable>
- </literallayout>
- </para>
- <para>
- An RPN pattern is a simple attribute list. Each attribute pair
- takes the form:
- <literallayout>
- [<replaceable>set</replaceable>] <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>
- </literallayout>
- The attribute <replaceable>set</replaceable> is optional.
- The <replaceable>type</replaceable> is the attribute type,
- <replaceable>value</replaceable> the attribute value.
- </para>
- <para>
- The character <literal>*</literal> (asterisk) has special meaning
- when used in the RPN pattern.
- Each occurrence of <literal>*</literal> is substituted with the
- CQL matching name (index, relation, qualifier etc).
- This facility can be used to copy a CQL name verbatim to the RPN result.
- </para>
- <para>
- The following CQL patterns are recognized:
- <variablelist>
- <varlistentry><term>
- <literal>index.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
- </term>
- <listitem>
- <para>
- This pattern is invoked when a CQL index, such as
- dc.title is converted. <replaceable>set</replaceable>
- and <replaceable>name</replaceable> are the context set and index
- name respectively.
- Typically, the RPN specifies an equivalent use attribute.
- </para>
- <para>
- For terms not bound by an index the pattern
- <literal>index.cql.serverChoice</literal> is used.
- Here, the prefix <literal>cql</literal> is defined as
- <literal>http://www.loc.gov/zing/cql/cql-indexes/v1.0/</literal>.
- If this pattern is not defined, the mapping will fail.
- </para>
- <para>
- The pattern,
- <literal>index.</literal><replaceable>set</replaceable><literal>.*</literal>
- is used when no other index pattern is matched.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry><term>
- <literal>qualifier.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
- (DEPRECATED)
- </term>
- <listitem>
- <para>
- For backwards compatibility, this is recognised as a synonym of
- <literal>index.</literal><replaceable>set</replaceable><literal>.</literal><replaceable>name</replaceable>
- </para>
- </listitem>
- </varlistentry>
- <varlistentry><term>
- <literal>relation.</literal><replaceable>relation</replaceable>
- </term>
- <listitem>
- <para>
- This pattern specifies how a CQL relation is mapped to RPN.
- <replaceable>pattern</replaceable> is name of relation
- operator. Since <literal>=</literal> is used as
- separator between CQL pattern and RPN, CQL relations
- including <literal>=</literal> cannot be
- used directly. To avoid a conflict, the names
- <literal>ge</literal>,
- <literal>eq</literal>,
- <literal>le</literal>,
- must be used for CQL operators, greater-than-or-equal,
- equal, less-than-or-equal respectively.
- The RPN pattern is supposed to include a relation attribute.
- </para>
- <para>
- For terms not bound by a relation, the pattern
- <literal>relation.scr</literal> is used. If the pattern
- is not defined, the mapping will fail.
- </para>
- <para>
- The special pattern, <literal>relation.*</literal> is used
- when no other relation pattern is matched.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>relationModifier.</literal><replaceable>mod</replaceable>
- </term>
- <listitem>
- <para>
- This pattern specifies how a CQL relation modifier is mapped to RPN.
- The RPN pattern is usually a relation attribute.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>structure.</literal><replaceable>type</replaceable>
- </term>
- <listitem>
- <para>
- This pattern specifies how a CQL structure is mapped to RPN.
- Note that this CQL pattern is somewhat to similar to
- CQL pattern <literal>relation</literal>.
- The <replaceable>type</replaceable> is a CQL relation.
- </para>
- <para>
- The pattern, <literal>structure.*</literal> is used
- when no other structure pattern is matched.
- Usually, the RPN equivalent specifies a structure attribute.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>position.</literal><replaceable>type</replaceable>
- </term>
- <listitem>
- <para>
- This pattern specifies how the anchor (position) of
- CQL is mapped to RPN.
- The <replaceable>type</replaceable> is one
- of <literal>first</literal>, <literal>any</literal>,
- <literal>last</literal>, <literal>firstAndLast</literal>.
- </para>
- <para>
- The pattern, <literal>position.*</literal> is used
- when no other position pattern is matched.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>set.</literal><replaceable>prefix</replaceable>
- </term>
- <listitem>
- <para>
- This specification defines a CQL context set for a given prefix.
- The value on the right hand side is the URI for the set -
- <emphasis>not</emphasis> RPN. All prefixes used in
- index patterns must be defined this way.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term>
- <literal>set</literal>
- </term>
- <listitem>
- <para>
- This specification defines a default CQL context set for index names.
- The value on the right hand side is the URI for the set.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </para>
- <example id="example.cql.to.rpn.mapping">
- <title>CQL to RPN mapping file</title>
- <para>
- This simple file defines two context sets, three indexes and three
- relations, a position pattern and a default structure.
- </para>
- <programlisting><![CDATA[
- set.cql = http://www.loc.gov/zing/cql/context-sets/cql/v1.1/
- set.dc = http://www.loc.gov/zing/cql/dc-indexes/v1.0/
-
- index.cql.serverChoice = 1=1016
- index.dc.title = 1=4
- index.dc.subject = 1=21
-
- relation.< = 2=1
- relation.eq = 2=3
- relation.scr = 2=3
-
- position.any = 3=3 6=1
-
- structure.* = 4=1
-]]>
- </programlisting>
- <para>
- With the mappings above, the CQL query
- <screen>
- computer
- </screen>
- is converted to the PQF:
- <screen>
- @attr 1=1016 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "computer"
- </screen>
- by rules <literal>index.cql.serverChoice</literal>,
- <literal>relation.scr</literal>, <literal>structure.*</literal>,
- <literal>position.any</literal>.
- </para>
- <para>
- CQL query
- <screen>
- computer^
- </screen>
- is rejected, since <literal>position.right</literal> is
- undefined.
- </para>
- <para>
- CQL query
- <screen>
- >my = "http://www.loc.gov/zing/cql/dc-indexes/v1.0/" my.title = x
- </screen>
- is converted to
- <screen>
- @attr 1=4 @attr 2=3 @attr 4=1 @attr 3=3 @attr 6=1 "x"
- </screen>
- </para>
- </example>
- <example id="example.cql.to.rpn.string">
- <title>CQL to RPN string attributes</title>
- <para>
- In this example we allow any index to be passed to RPN as
- a use attribute.
- </para>
- <programlisting><![CDATA[
- # Identifiers for prefixes used in this file. (index.*)
- set.cql = info:srw/cql-context-set/1/cql-v1.1
- set.rpn = http://bogus/rpn
- set = http://bogus/rpn
-
- # The default index when none is specified by the query
- index.cql.serverChoice = 1=any
-
- index.rpn.* = 1=*
- relation.eq = 2=3
- structure.* = 4=1
- position.any = 3=3
-]]>
- </programlisting>
- <para>
- The <literal>http://bogus/rpn</literal> context set is also the default
- so we can make queries such as
- <screen>
- title = a
- </screen>
- which is converted to
- <screen>
- @attr 2=3 @attr 4=1 @attr 3=3 @attr 1=title "a"
- </screen>
- </para>
- </example>
- <example id="example.cql.to.rpn.bathprofile">
- <title>CQL to RPN using Bath Profile</title>
- <para>
- The file <filename>etc/pqf.properties</filename> has mappings from
- the Bath Profile and Dublin Core to RPN.
- If YAZ is installed as a package it's usually located
- in <filename>/usr/share/yaz/etc</filename> and part of the
- development package, such as <literal>libyaz-dev</literal>.
- </para>
- </example>
- </sect3>
- <sect3 id="cql.xcql"><title>CQL to XCQL conversion</title>
- <para>
- Conversion from CQL to XCQL is trivial and does not
- require a mapping to be defined.
- There three functions to choose from depending on the
- way you wish to store the resulting output (XML buffer
- containing XCQL).
- <synopsis>
-int cql_to_xml_buf(struct cql_node *cn, char *out, int max);
-void cql_to_xml(struct cql_node *cn,
- void (*pr)(const char *buf, void *client_data),
- void *client_data);
-void cql_to_xml_stdio(struct cql_node *cn, FILE *f);
- </synopsis>
- Function <function>cql_to_xml_buf</function> converts
- to XCQL and stores result in a user supplied buffer of a given
- max size.
- </para>
- <para>
- <function>cql_to_xml</function> writes the result in
- a user defined output stream.
- <function>cql_to_xml_stdio</function> writes to a
- a file.
- </para>
- </sect3>
- <sect3 id="rpn.to.cql">
- <title>PQF to CQL conversion</title>
- <para>
- Conversion from PQF to CQL is offered by the two functions shown
- below. The former uses a generic stream for result. The latter
- puts result in a WRBUF (string container).
- <synopsis>
-#include <yaz/rpn2cql.h>
-
-int cql_transform_rpn2cql_stream(cql_transform_t ct,
- void (*pr)(const char *buf, void *client_data),
- void *client_data,
- Z_RPNQuery *q);
-
-int cql_transform_rpn2cql_wrbuf(cql_transform_t ct,
- WRBUF w,
- Z_RPNQuery *q);
- </synopsis>
- The configuration is the same as used in CQL to PQF conversions.
- </para>
- </sect3>
- </sect2>
- </sect1>
- <sect1 id="tools.oid"><title>Object Identifiers</title>
-
- <para>
- The basic YAZ representation of an OID is an array of integers,
- terminated with the value -1. This integer is of type
- <literal>Odr_oid</literal>.
- </para>
- <para>
- Fundamental OID operations and the type <literal>Odr_oid</literal>
- are defined in <filename>yaz/oid_util.h</filename>.
- </para>
- <para>
- An OID can either be declared as a automatic variable or it can
- allocated using the memory utilities or ODR/NMEM. It's
- guaranteed that an OID can fit in <literal>OID_SIZE</literal> integers.
- </para>
- <example id="tools.oid.bib1.1"><title>Create OID on stack</title>
- <para>
- We can create an OID for the Bib-1 attribute set with:
- <screen>
- Odr_oid bib1[OID_SIZE];
- bib1[0] = 1;
- bib1[1] = 2;
- bib1[2] = 840;
- bib1[3] = 10003;
- bib1[4] = 3;
- bib1[5] = 1;
- bib1[6] = -1;
- </screen>
- </para>
- </example>
- <para>
- And OID may also be filled from a string-based representation using
- dots (.). This is achieved by function
- <screen>
- int oid_dotstring_to_oid(const char *name, Odr_oid *oid);
- </screen>
- This functions returns 0 if name could be converted; -1 otherwise.
- </para>
- <example id="tools.oid.bib1.2"><title>Using oid_oiddotstring_to_oid</title>
- <para>
- We can fill the Bib-1 attribute set OID easier with:
- <screen>
- Odr_oid bib1[OID_SIZE];
- oid_oiddotstring_to_oid("1.2.840.10003.3.1", bib1);
- </screen>
- </para>
- </example>
- <para>
- We can also allocate an OID dynamically on a ODR stream with:
- <screen>
- Odr_oid *odr_getoidbystr(ODR o, const char *str);
- </screen>
- This creates an OID from string-based representation using dots.
- This function take an &odr; stream as parameter. This stream is used to
- allocate memory for the data elements, which is released on a
- subsequent call to <function>odr_reset()</function> on that stream.
- </para>
-
- <example id="tools.oid.bib1.3"><title>Using odr_getoidbystr</title>
- <para>
- We can create a OID for the Bib-1 attribute set with:
- <screen>
- Odr_oid *bib1 = odr_getoidbystr(odr, "1.2.840.10003.3.1");
- </screen>
- </para>
- </example>
-
- <para>
- The function
- <screen>
- char *oid_oid_to_dotstring(const Odr_oid *oid, char *oidbuf)
- </screen>
- does the reverse of <function>oid_oiddotstring_to_oid</function>. It
- converts an OID to the string-based representation using dots.
- The supplied char buffer <literal>oidbuf</literal> holds the resulting
- string and must be at least <literal>OID_STR_MAX</literal> in size.
- </para>
-
- <para>
- OIDs can be copied with <function>oid_oidcpy</function> which takes
- two OID lists as arguments. Alternativly, an OID copy can be allocated
- on a ODR stream with:
- <screen>
- Odr_oid *odr_oiddup(ODR odr, const Odr_oid *o);
- </screen>
- </para>
-
- <para>
- OIDs can be compared with <function>oid_oidcmp</function> which returns
- zero if the two OIDs provided are identical; non-zero otherwise.
- </para>
-
- <sect2 id="tools.oid.database"><title>OID database</title>
- <para>
- From YAZ version 3 and later, the oident system has been replaced
- by an OID database. OID database is a misnomer .. the old odient
- system was also a database.
- </para>
- <para>
- The OID database is really just a map between named Object Identifiers
- (string) and their OID raw equivalents. Most operations either
- convert from string to OID or other way around.
- </para>
- <para>
- Unfortunately, whenever we supply a string we must also specify the
- <emphasis>OID class</emphasis>. The class is necessary because some
- strings correspond to multiple OIDs. An example of such a string is
- <literal>Bib-1</literal> which may either be an attribute-set
- or a diagnostic-set.
- </para>
- <para>
- Applications using the YAZ database should include
- <filename>yaz/oid_db.h</filename>.
- </para>
- <para>
- A YAZ database handle is of type <literal>yaz_oid_db_t</literal>.
- Actually that's a pointer. You need not think deal with that.
- YAZ has a built-in database which can be considered "constant" for
- most purposes.
- We can get hold that by using function <function>yaz_oid_std</function>.
- </para>
- <para>
- All functions with prefix <function>yaz_string_to_oid</function>
- converts from class + string to OID. We have variants of this
- operation due to different memory allocation strategies.
- </para>
- <para>
- All functions with prefix
- <function>yaz_oid_to_string</function> converts from OID to string
- + class.
- </para>
-
- <example id="tools.oid.bib1.4"><title>Create OID with YAZ DB</title>
- <para>
- We can create an OID for the Bib-1 attribute set on the ODR stream
- odr with:
- <screen>
- Odr_oid *bib1 =
- yaz_string_to_oid_odr(yaz_oid_std(), CLASS_ATTSET, "Bib-1", odr);
- </screen>
- This is more complex than using <function>odr_getoidbystr</function>.
- You would only use <function>yaz_string_to_oid_odr</function> when the
- string (here Bib-1) is supplied by a user or configuration.
- </para>
- </example>
-
- </sect2>
- <sect2 id="tools.oid.std"><title>Standard OIDs</title>
-
- <para>
- All the object identifers in the standard OID database as returned
- by <function>yaz_oid_std</function> can referenced directly in a
- program as a constant OID.
- Each constant OID is prefixed with <literal>yaz_oid_</literal> -
- followed by OID class (lowercase) - then by OID name (normalized and
- lowercase).
- </para>
- <para>
- See <xref linkend="list-oids"/> for list of all object identifiers
- built into YAZ.
- These are declared in <filename>yaz/oid_std.h</filename> but are
- included by <filename>yaz/oid_db.h</filename> as well.
- </para>
-
- <example id="tools.oid.bib1.5"><title>Use a built-in OID</title>
- <para>
- We can allocate our own OID filled with the constant OID for
- Bib-1 with:
- <screen>
- Odr_oid *bib1 = odr_oiddup(o, yaz_oid_attset_bib1);
- </screen>
- </para>
- </example>
- </sect2>
- </sect1>
- <sect1 id="tools.nmem"><title>Nibble Memory</title>
-
- <para>
- Sometimes when you need to allocate and construct a large,
- interconnected complex of structures, it can be a bit of a pain to
- release the associated memory again. For the structures describing the
- Z39.50 PDUs and related structures, it is convenient to use the
- memory-management system of the &odr; subsystem (see
- <xref linkend="odr.use"/>). However, in some circumstances
- where you might otherwise benefit from using a simple nibble memory
- management system, it may be impractical to use
- <function>odr_malloc()</function> and <function>odr_reset()</function>.
- For this purpose, the memory manager which also supports the &odr;
- streams is made available in the NMEM module. The external interface
- to this module is given in the <filename>nmem.h</filename> file.
- </para>
-
- <para>
- The following prototypes are given:
- </para>
-
- <screen>
- NMEM nmem_create(void);
- void nmem_destroy(NMEM n);
- void *nmem_malloc(NMEM n, size_t size);
- void nmem_reset(NMEM n);
- size_t nmem_total(NMEM n);
- void nmem_init(void);
- void nmem_exit(void);
- </screen>
-
- <para>
- The <function>nmem_create()</function> function returns a pointer to a
- memory control handle, which can be released again by
- <function>nmem_destroy()</function> when no longer needed.
- The function <function>nmem_malloc()</function> allocates a block of
- memory of the requested size. A call to <function>nmem_reset()</function>
- or <function>nmem_destroy()</function> will release all memory allocated
- on the handle since it was created (or since the last call to
- <function>nmem_reset()</function>. The function
- <function>nmem_total()</function> returns the number of bytes currently
- allocated on the handle.
- </para>
-
- <para>
- The nibble memory pool is shared amongst threads. POSIX
- mutex'es and WIN32 Critical sections are introduced to keep the
- module thread safe. Function <function>nmem_init()</function>
- initializes the nibble memory library and it is called automatically
- the first time the <literal>YAZ.DLL</literal> is loaded. &yaz; uses
- function <function>DllMain</function> to achieve this. You should
- <emphasis>not</emphasis> call <function>nmem_init</function> or
- <function>nmem_exit</function> unless you're absolute sure what
- you're doing. Note that in previous &yaz; versions you'd have to call
- <function>nmem_init</function> yourself.
- </para>
-
- </sect1>
-
- <sect1 id="tools.log"><title>Log</title>
- <para>
- &yaz; has evolved a fairly complex log system which should be useful both
- for debugging &yaz; itself, debugging applications that use &yaz;, and for
- production use of those applications.
- </para>
- <para>
- The log functions are declared in header <filename>yaz/log.h</filename>
- and implemented in <filename>src/log.c</filename>.
- Due to name clash with syslog and some math utilities the logging
- interface has been modified as of YAZ 2.0.29. The obsolete interface
- is still available if in header file <filename>yaz/log.h</filename>.
- The key points of the interface are:
- </para>
- <screen>
- void yaz_log(int level, const char *fmt, ...)
-
- void yaz_log_init(int level, const char *prefix, const char *name);
- void yaz_log_init_file(const char *fname);
- void yaz_log_init_level(int level);
- void yaz_log_init_prefix(const char *prefix);
- void yaz_log_time_format(const char *fmt);
- void yaz_log_init_max_size(int mx);
-
- int yaz_log_mask_str(const char *str);
- int yaz_log_module_level(const char *name);
- </screen>
-
- <para>
- The reason for the whole log module is the <function>yaz_log</function>
- function. It takes a bitmask indicating the log levels, a
- <literal>printf</literal>-like format string, and a variable number of
- arguments to log.
- </para>
-
- <para>
- The <literal>log level</literal> is a bit mask, that says on which level(s)
- the log entry should be made, and optionally set some behaviour of the
- logging. In the most simple cases, it can be one of <literal>YLOG_FATAL,
- YLOG_DEBUG, YLOG_WARN, YLOG_LOG</literal>. Those can be combined with bits
- that modify the way the log entry is written:<literal>YLOG_ERRNO,
- YLOG_NOTIME, YLOG_FLUSH</literal>.
- Most of the rest of the bits are deprecated, and should not be used. Use
- the dynamic log levels instead.
- </para>
-
- <para>
- Applications that use &yaz;, should not use the LOG_LOG for ordinary
- messages, but should make use of the dynamic loglevel system. This consists
- of two parts, defining the loglevel and checking it.
- </para>
-
- <para>
- To define the log levels, the (main) program should pass a string to
- <function>yaz_log_mask_str</function> to define which log levels are to be
- logged. This string should be a comma-separated list of log level names,
- and can contain both hard-coded names and dynamic ones. The log level
- calculation starts with <literal>YLOG_DEFAULT_LEVEL</literal> and adds a bit
- for each word it meets, unless the word starts with a '-', in which case it
- clears the bit. If the string <literal>'none'</literal> is found,
- all bits are cleared. Typically this string comes from the command-line,
- often identified by <literal>-v</literal>. The
- <function>yaz_log_mask_str</function> returns a log level that should be
- passed to <function>yaz_log_init_level</function> for it to take effect.
- </para>
-
- <para>
- Each module should check what log bits it should be used, by calling
- <function>yaz_log_module_level</function> with a suitable name for the
- module. The name is cleared from a preceding path and an extension, if any,
- so it is quite possible to use <literal>__FILE__</literal> for it. If the
- name has been passed to <function>yaz_log_mask_str</function>, the routine
- returns a non-zero bitmask, which should then be used in consequent calls
- to yaz_log. (It can also be tested, so as to avoid unnecessary calls to
- yaz_log, in time-critical places, or when the log entry would take time
- to construct.)
- </para>
-
- <para>
- Yaz uses the following dynamic log levels:
- <literal>server, session, request, requestdetail</literal> for the server
- functionality.
- <literal>zoom</literal> for the zoom client api.
- <literal>ztest</literal> for the simple test server.
- <literal>malloc, nmem, odr, eventl</literal> for internal debugging of yaz itself.
- Of course, any program using yaz is welcome to define as many new ones, as
- it needs.
- </para>
-
- <para>
- By default the log is written to stderr, but this can be changed by a call
- to <function>yaz_log_init_file</function> or
- <function>yaz_log_init</function>. If the log is directed to a file, the
- file size is checked at every write, and if it exceeds the limit given in
- <function>yaz_log_init_max_size</function>, the log is rotated. The
- rotation keeps one old version (with a <literal>.1</literal> appended to
- the name). The size defaults to 1GB. Setting it to zero will disable the
- rotation feature.
- </para>
-
- <screen>
- A typical yaz-log looks like this
- 13:23:14-23/11 yaz-ztest(1) [session] Starting session from tcp:127.0.0.1 (pid=30968)
- 13:23:14-23/11 yaz-ztest(1) [request] Init from 'YAZ' (81) (ver 2.0.28) OK
- 13:23:17-23/11 yaz-ztest(1) [request] Search Z: @attrset Bib-1 foo OK:7 hits
- 13:23:22-23/11 yaz-ztest(1) [request] Present: [1] 2+2 OK 2 records returned
- 13:24:13-23/11 yaz-ztest(1) [request] Close OK
- </screen>
-
- <para>
- The log entries start with a time stamp. This can be omitted by setting the
- <literal>YLOG_NOTIME</literal> bit in the loglevel. This way automatic tests
- can be hoped to produce identical log files, that are easy to diff. The
- format of the time stamp can be set with
- <function>yaz_log_time_format</function>, which takes a format string just
- like <function>strftime</function>.
- </para>
-
- <para>
- Next in a log line comes the prefix, often the name of the program. For
- yaz-based servers, it can also contain the session number. Then
- comes one or more logbits in square brackets, depending on the logging
- level set by <function>yaz_log_init_level</function> and the loglevel
- passed to <function>yaz_log_init_level</function>. Finally comes the format
- string and additional values passed to <function>yaz_log</function>
- </para>
-
- <para>
- The log level <literal>YLOG_LOGLVL</literal>, enabled by the string
- <literal>loglevel</literal>, will log all the log-level affecting
- operations. This can come in handy if you need to know what other log
- levels would be useful. Grep the logfile for <literal>[loglevel]</literal>.
- </para>
-
- <para>
- The log system is almost independent of the rest of &yaz;, the only
- important dependence is of <filename>nmem</filename>, and that only for
- using the semaphore definition there.
- </para>
-
- <para>
- The dynamic log levels and log rotation were introduced in &yaz; 2.0.28. At
- the same time, the log bit names were changed from
- <literal>LOG_something</literal> to <literal>YLOG_something</literal>,
- to avoid collision with <filename>syslog.h</filename>.
- </para>
-
- </sect1>
-
- <sect1 id="marc"><title>MARC</title>
-
- <para>
- YAZ provides a fast utility for working with MARC records.
- Early versions of the MARC utility only allowed decoding of ISO2709.
- Today the utility may both encode - and decode to a varity of formats.
- </para>
- <synopsis><![CDATA[
- #include <yaz/marcdisp.h>
-
- /* create handler */
- yaz_marc_t yaz_marc_create(void);
- /* destroy */
- void yaz_marc_destroy(yaz_marc_t mt);
-
- /* set XML mode YAZ_MARC_LINE, YAZ_MARC_SIMPLEXML, ... */
- void yaz_marc_xml(yaz_marc_t mt, int xmlmode);
- #define YAZ_MARC_LINE 0
- #define YAZ_MARC_SIMPLEXML 1
- #define YAZ_MARC_OAIMARC 2
- #define YAZ_MARC_MARCXML 3
- #define YAZ_MARC_ISO2709 4
- #define YAZ_MARC_XCHANGE 5
- #define YAZ_MARC_CHECK 6
- #define YAZ_MARC_TURBOMARC 7
- #define YAZ_MARC_JSON 8
-
- /* supply iconv handle for character set conversion .. */
- void yaz_marc_iconv(yaz_marc_t mt, yaz_iconv_t cd);
-
- /* set debug level, 0=none, 1=more, 2=even more, .. */
- void yaz_marc_debug(yaz_marc_t mt, int level);
-
- /* decode MARC in buf of size bsize. Returns >0 on success; <=0 on failure.
- On success, result in *result with size *rsize. */
- int yaz_marc_decode_buf(yaz_marc_t mt, const char *buf, int bsize,
- const char **result, size_t *rsize);
-
- /* decode MARC in buf of size bsize. Returns >0 on success; <=0 on failure.
- On success, result in WRBUF */
- int yaz_marc_decode_wrbuf(yaz_marc_t mt, const char *buf,
- int bsize, WRBUF wrbuf);
-]]>
- </synopsis>
- <note>
- <para>
- The synopsis is just a basic subset of all functionality. Refer
- to the actual header file <filename>marcdisp.h</filename> for
- details.
- </para>
- </note>
- <para>
- A MARC conversion handle must be created by using
- <function>yaz_marc_create</function> and destroyed
- by calling <function>yaz_marc_destroy</function>.
- </para>
- <para>
- All other function operate on a <literal>yaz_marc_t</literal> handle.
- The output is specified by a call to <function>yaz_marc_xml</function>.
- The <literal>xmlmode</literal> must be one of
- <variablelist>
- <varlistentry>
- <term>YAZ_MARC_LINE</term>
- <listitem>
- <para>
- A simple line-by-line format suitable for display but not
- recommend for further (machine) processing.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>YAZ_MARC_MARCXML</term>
- <listitem>
- <para>
- <ulink url="&url.marcxml;">MARCXML</ulink>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>YAZ_MARC_ISO2709</term>
- <listitem>
- <para>
- ISO2709 (sometimes just referred to as "MARC").
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>YAZ_MARC_XCHANGE</term>
- <listitem>
- <para>
- <ulink url="&url.marcxchange;">MarcXchange</ulink>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>YAZ_MARC_CHECK</term>
- <listitem>
- <para>
- Pseudo format for validation only. Does not generate
- any real output except diagnostics.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>YAZ_MARC_TURBOMARC</term>
- <listitem>
- <para>
- XML format with same semantics as MARCXML but more compact
- and geared towards fast processing with XSLT. Refer to
- <xref linkend="tools.turbomarc"/> for more information.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry>
- <term>YAZ_MARC_JSON</term>
- <listitem>
- <para>
- <ulink url="&url.marc_in_json;">MARC-in_JSON</ulink> format.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </para>
- <para>
- The actual conversion functions are
- <function>yaz_marc_decode_buf</function> and
- <function>yaz_marc_decode_wrbuf</function> which decodes and encodes
- a MARC record. The former function operates on simple buffers, the
- stores the resulting record in a WRBUF handle (WRBUF is a simple string
- type).
- </para>
- <example id="example.marc.display">
- <title>Display of MARC record</title>
- <para>
- The following program snippet illustrates how the MARC API may
- be used to convert a MARC record to the line-by-line format:
- <programlisting><![CDATA[
- void print_marc(const char *marc_buf, int marc_buf_size)
- {
- char *result; /* for result buf */
- size_t result_len; /* for size of result */
- yaz_marc_t mt = yaz_marc_create();
- yaz_marc_xml(mt, YAZ_MARC_LINE);
- yaz_marc_decode_buf(mt, marc_buf, marc_buf_size,
- &result, &result_len);
- fwrite(result, result_len, 1, stdout);
- yaz_marc_destroy(mt); /* note that result is now freed... */
- }
-]]>
- </programlisting>
- </para>
- </example>
- <sect2 id="tools.turbomarc">
- <title>TurboMARC</title>
- <para>
- TurboMARC is yet another XML encoding of a MARC record. The format
- was designed for fast processing with XSLT.
- </para>
- <para>
- Applications like
- Pazpar2 uses XSLT to convert an XML encoded MARC record to an internal
- representation. This conversion mostly check the tag of a MARC field
- to determine the basic rules in the conversion. This check is
- costly when that is tag is encoded as an attribute in MARCXML.
- By having the tag value as the element instead, makes processing
- many times faster (at least for Libxslt).
- </para>
- <para>
- TurboMARC is encoded as follows:
- <itemizedlist>
- <listitem><para>
- Record elements is part of namespace
- "<literal>http://www.indexdata.com/turbomarc</literal>".
- </para></listitem>
- <listitem><para>
- A record is enclosed in element <literal>r</literal>.
- </para></listitem>
- <listitem><para>
- A collection of records is enclosed in element
- <literal>collection</literal>.
- </para></listitem>
- <listitem><para>
- The leader is encoded as element <literal>l</literal> with the
- leader content as its (text) value.
- </para></listitem>
- <listitem><para>
- A control field is encoded as element <literal>c</literal> concatenated
- with the tag value of the control field if the tag value
- matches the regular expression <literal>[a-zA-Z0-9]*</literal>.
- If the tag value do not match the regular expression
- <literal>[a-zA-Z0-9]*</literal> the control field is encoded
- as element <literal>c</literal> and attribute <literal>code</literal>
- will hold the tag value.
- This rule ensure that in the rare cases where a tag value might
- result in a non-wellformed XML YAZ encode it as a coded attribute
- (as in MARCXML).
- </para>
- <para>
- The control field content is the the text value of this element.
- Indicators are encoded as attribute names
- <literal>i1</literal>, <literal>i2</literal>, etc.. and
- corresponding values for each indicator.
- </para></listitem>
- <listitem><para>
- A data field is encoded as element <literal>d</literal> concatenated
- with the tag value of the data field or using the attribute
- <literal>code</literal> as described in the rules for control fields.
- The children of the data field element is subfield elements.
- Each subfield element is encoded as <literal>s</literal>
- concatenated with the sub field code.
- The text of the subfield element is the contents of the subfield.
- Indicators are encoded as attributes for the data field element similar
- to the encoding for control fields.
- </para></listitem>
- </itemizedlist>
- </para>
- </sect2>
- </sect1>
-
- <sect1 id="tools.retrieval">
- <title>Retrieval Facility</title>
- <para>
- YAZ version 2.1.20 or later includes a Retrieval facility tool
- which allows a SRU/Z39.50 to describe itself and perform record
- conversions. The idea is the following:
-
- <itemizedlist>
- <listitem>
- <para>
- An SRU/Z39.50 client sends a retrieval request which includes
- a combination of the following parameters: syntax (format),
- schema (or element set name).
- </para>
- </listitem>
-
- <listitem>
- <para>
- The retrieval facility is invoked with parameters in a
- server/proxy. The retrieval facility matches the parameters a set of
- "supported" retrieval types.
- If there is no match, the retrieval signals an error
- (syntax and / or schema not supported).
- </para>
- </listitem>
-
- <listitem>
- <para>
- For a successful match, the backend is invoked with the same
- or altered retrieval parameters (syntax, schema). If
- a record is received from the backend, it is converted to the
- frontend name / syntax.
- </para>
- </listitem>
-
- <listitem>
- <para>
- The resulting record is sent back the client and tagged with
- the frontend syntax / schema.
- </para>
- </listitem>
-
- </itemizedlist>
- </para>
- <para>
- The Retrieval facility is driven by an XML configuration. The
- configuration is neither Z39.50 ZeeRex or SRU ZeeRex. But it
- should be easy to generate both of them from the XML configuration.
- (unfortunately the two versions
- of ZeeRex differ substantially in this regard).
- </para>
- <sect2 id="tools.retrieval.format">
- <title>Retrieval XML format</title>
- <para>
- All elements should be covered by namespace
- <literal>http://indexdata.com/yaz</literal> .
- The root element node must be <literal>retrievalinfo</literal>.
- </para>
- <para>
- The <literal>retrievalinfo</literal> must include one or
- more <literal>retrieval</literal> elements. Each
- <literal>retrieval</literal> defines specific combination of
- syntax, name and identifier supported by this retrieval service.
- </para>
- <para>
- The <literal>retrieval</literal> element may include any of the
- following attributes:
- <variablelist>
- <varlistentry><term><literal>syntax</literal> (REQUIRED)</term>
- <listitem>
- <para>
- Defines the record syntax. Possible values is any
- of the names defined in YAZ' OID database or a raw
- OID in (n.n ... n).
- </para>
- </listitem>
- </varlistentry>
- <varlistentry><term><literal>name</literal> (OPTIONAL)</term>
- <listitem>
- <para>
- Defines the name of the retrieval format. This can be
- any string. For SRU, the value, is equivalent to schema (short-hand);
- for Z39.50 it's equivalent to simple element set name.
- For YAZ 3.0.24 and later this name may be specified as a glob
- expression with operators
- <literal>*</literal> and <literal>?</literal>.
- </para>
- </listitem>
- </varlistentry>
- <varlistentry><term><literal>identifier</literal> (OPTIONAL)</term>
- <listitem>
- <para>
- Defines the URI schema name of the retrieval format. This can be
- any string. For SRU, the value, is equivalent to URI schema.
- For Z39.50, there is no equivalent.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
- <para>
- The <literal>retrieval</literal> may include one
- <literal>backend</literal> element. If a <literal>backend</literal>
- element is given, it specifies how the records are retrieved by
- some backend and how the records are converted from the backend to
- the "frontend".
- </para>
- <para>
- The attributes, <literal>name</literal> and <literal>syntax</literal>
- may be specified for the <literal>backend</literal> element. These
- semantics of these attributes is equivalent to those for the
- <literal>retrieval</literal>. However, these values are passed to
- the "backend".
- </para>
- <para>
- The <literal>backend</literal> element may includes one or more
- conversion instructions (as children elements). The supported
- conversions are:
- <variablelist>
- <varlistentry><term><literal>marc</literal></term>
- <listitem>
- <para>
- The <literal>marc</literal> element specifies a conversion
- to - and from ISO2709 encoded MARC and
- <ulink url="&url.marcxml;">&acro.marcxml;</ulink>/MarcXchange.
- The following attributes may be specified:
-
- <variablelist>
- <varlistentry><term><literal>inputformat</literal> (REQUIRED)</term>
- <listitem>
- <para>
- Format of input. Supported values are
- <literal>marc</literal> (for ISO2709), <literal>xml</literal>
- (MARCXML/MarcXchange) and <literal>json</literal>
- (<ulink url="&url.marc_in_json;">MARC-in_JSON</ulink>).
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term><literal>outputformat</literal> (REQUIRED)</term>
- <listitem>
- <para>
- Format of output. Supported values are
- <literal>line</literal> (MARC line format);
- <literal>marcxml</literal> (for MARCXML),
- <literal>marc</literal> (ISO2709),
- <literal>marcxhcange</literal> (for MarcXchange),
- or <literal>json</literal>
- (<ulink url="&url.marc_in_json;">MARC-in_JSON </ulink>).
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term><literal>inputcharset</literal> (OPTIONAL)</term>
- <listitem>
- <para>
- Encoding of input. For XML input formats, this need not
- be given, but for ISO2709 based inputformats, this should
- be set to the encoding used. For MARC21 records, a common
- inputcharset value would be <literal>marc-8</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term><literal>outputcharset</literal> (OPTIONAL)</term>
- <listitem>
- <para>
- Encoding of output. If outputformat is XML based, it is
- strongly recommened to use <literal>utf-8</literal>.
- </para>
- </listitem>
- </varlistentry>
-
- </variablelist>
- </para>
- </listitem>
- </varlistentry>
- <varlistentry><term><literal>xslt</literal></term>
- <listitem>
- <para>
- The <literal>xslt</literal> element specifies a conversion
- via &acro.xslt;. The following attributes may be specified:
-
- <variablelist>
- <varlistentry><term><literal>stylesheet</literal> (REQUIRED)</term>
- <listitem>
- <para>
- Stylesheet file.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
-
- </para>
- </listitem>
- </varlistentry>
-
- <varlistentry><term><literal>solrmarc</literal></term>
- <listitem>
- <para>
- The <literal>solrmarc</literal> decodes solrmarc records.
- It assumes that the input is pure solrmarc text (no escaping)
- and will convert all sequences of the form #XX; to a single
- character of the hexadecimal value as given by XX. The output,
- presumably, is a valid ISO2709 buffer.
- </para>
- <para>
- This conversion is available in YAZ 5.0.21 and later.
- </para>
- </listitem>
- </varlistentry>
- </variablelist>
- </para>
- </sect2>
- <sect2 id="tools.retrieval.examples">
- <title>Retrieval Facility Examples</title>
- <example id="tools.retrieval.marc21">
- <title>MARC21 backend</title>
- <para>
- A typical way to use the retrieval facility is to enable XML
- for servers that only supports ISO2709 encoded MARC21 records.
- </para>
- <programlisting><![CDATA[
- <retrievalinfo>
- <retrieval syntax="usmarc" name="F"/>
- <retrieval syntax="usmarc" name="B"/>
- <retrieval syntax="xml" name="marcxml"
- identifier="info:srw/schema/1/marcxml-v1.1">
- <backend syntax="usmarc" name="F">
- <marc inputformat="marc" outputformat="marcxml"
- inputcharset="marc-8"/>
- </backend>
- </retrieval>
- <retrieval syntax="xml" name="dc">
- <backend syntax="usmarc" name="F">
- <marc inputformat="marc" outputformat="marcxml"
- inputcharset="marc-8"/>
- <xslt stylesheet="MARC21slim2DC.xsl"/>
- </backend>
- </retrieval>
- </retrievalinfo>
-]]>
- </programlisting>
- <para>
- This means that our frontend supports:
- <itemizedlist>
- <listitem>
- <para>
- MARC21 F(ull) records.
- </para>
- </listitem>
- <listitem>
- <para>
- MARC21 B(rief) records.
- </para>
- </listitem>
-
- <listitem>
- <para>
- MARCXML records.
- </para>
- </listitem>
-
- <listitem>
- <para>
- Dublin core records.
- </para>
- </listitem>
- </itemizedlist>
- </para>
- </example>
-
- <example id="tools.retrieval.marcxml">
- <title>MARCXML backend</title>
- <para>
- SRW/SRU and Solr backends returns records in XML.
- If they return MARCXML or MarcXchange, the retrieval module
- can convert those into ISO2709 formats, most commonly USMARC
- (AKA MARC21).
- In this example, the backend returns MARCXML for schema="marcxml".
- </para>
- <programlisting><![CDATA[
- <retrievalinfo>
- <retrieval syntax="usmarc">
- <backend syntax="xml" name="marcxml">
- <marc inputformat="xml" outputformat="marc"
- outputcharset="marc-8"/>
- </backend>
- </retrieval>
- <retrieval syntax="xml" name="marcxml"
- identifier="info:srw/schema/1/marcxml-v1.1"/>
- <retrieval syntax="xml" name="dc">
- <backend syntax="xml" name="marcxml">
- <xslt stylesheet="MARC21slim2DC.xsl"/>
- </backend>
- </retrieval>
- </retrievalinfo>
-]]>
- </programlisting>
- <para>
- This means that our frontend supports:
- <itemizedlist>
- <listitem>
- <para>
- MARC21 records (any element set name) in MARC-8 encoding.
- </para>
- </listitem>
- <listitem>
- <para>
- MARCXML records for element-set=marcxml
- </para>
- </listitem>
- <listitem>
- <para>
- Dublin core records for element-set=dc.
- </para>
- </listitem>
- </itemizedlist>
- </para>
- </example>
-
- </sect2>
- <sect2 id="tools.retrieval.api">
- <title>API</title>
- <para>
- It should be easy to use the retrieval systems from applications. Refer
- to the headers
- <filename>yaz/retrieval.h</filename> and
- <filename>yaz/record_conv.h</filename>.
- </para>
- </sect2>
- </sect1>
- <sect1 id="sorting"><title>Sorting</title>
- <para>
- This chapter describes sorting and how it is supported in YAZ.
- Sorting applies to a result-set.
- The <ulink url="http://www.loc.gov/z3950/agency/markup/05.html#3.2.7">
- Z39.50 sorting facility
- </ulink>
- takes one or more input result-sets
- and one result-set as output. The most simple case is that
- the input-set is the same as the output-set.
- </para>
- <para>
- Z39.50 sorting has a separate APDU (service) that is, thus, performed
- following a search (two phases).
- </para>
- <para>
- In SRU/Solr, however, the model is different. Here, sorting is specified
- during the the search operation. Note, however, that SRU might
- perform sort as separate search, by referring to an existing result-set
- in the query (result-set reference).
- </para>
- <sect2><title>Using the Z39.50 sort service</title>
- <para>
- yaz-client and the ZOOM API supports the Z39.50 sort facility. In any
- case the sort sequence or sort critiera is using a string notation.
- This notation is a one-line notation suitable for being manually
- entered or generated and allows for easy logging (one liner).
- For the ZOOM API, the sort is specified in the call to ZOOM_query_sortby
- function. For yaz-client the sort is performed and specified using
- the sort and sort+ commands. For description of the sort criteria notation
- refer to the <link linkend="sortspec">sort command</link> in the
- yaz-client manual.
- </para>
- <para>
- The ZOOM API might choose one of several sort strategies for
- sorting. Refer to <xref linkend="zoom-sort-strategy"/>.
- </para>
- </sect2>
- <sect2><title>Type-7 sort</title>
- <para>
- Type-7 sort is an extension to the Bib-1 based RPN query where the
- sort specification is embedded as an Attribute-Plus-Term.
- </para>
- <para>
- The objectives for introducing Type-7 sorting is that it allows
- a client to perform sorting even if it does not implement/support
- Z39.50 sort. Virtually all Z39.50 client software supports
- RPN queries. It also may improve performance because the sort
- critieria is specified along with the search query.
- </para>
- <para>
- The sort is triggered by the presence of type 7 and the value of type 7
- specifies the
- <ulink url="http://www.loc.gov/z3950/agency/asn1.html#SortKeySpec">
- sortRelation
- </ulink>
- The value for type 7 is 1 for ascending and 2 for descending.
- For the
- <ulink url="http://www.loc.gov/z3950/agency/asn1.html#SortElement">
- sortElement
- </ulink>
- only the generic part is handled. If generic sortKey is of type
- sortField, then attribute type 1 is present and the value is
- sortField (InternationalString). If generic sortKey is of type
- sortAttributes, then the attributes in list is used . generic sortKey
- of type elementSpec is not supported.
- </para>
- <para>
- The term in the sorting Attribute-Plus-Term combo should hold
- an integer. The value is 0 for primary sorting criteria, 1 for second
- criteria, etc.
- </para>
- </sect2>
- </sect1>
- <sect1 id="facets"><title>Facets</title>
- <para>
- YAZ supports facets for in Solr, SRU 2.0 and Z39.50 protocols.
- </para>
- <para>
- Like Type-1/RPN, YAZ supports a string notation for specifying
- facets. For the API this is performed by
- <function>yaz_pqf_parse_facet_list</function>.
- </para>
- <para>
- For ZOOM C the facets are given by option "facets"
- For yaz-client it is used for the facets command.
- </para>
- <para>
- The grammar of this specification is as follows:
- <literallayout>
- facet-spec ::= facet-list
-
- facet-list ::= facet-list ',' attr-spec | attr-spec
-
- attr-spec ::= attr-spec '@attr' string | '@attr' string
-
- </literallayout>
- The notation is inspired by PQF. The string following '@attr'
- may not include blanks and is of the form
- <replaceable>type</replaceable><literal>=</literal><replaceable>value</replaceable>,
- where <replaceable>type</replaceable> is an integer and
- <replaceable>value</replaceable> is a string or an integer.
- </para>
- <para>
- The Facets specification is not Bib-1. The following types apply:
- </para>
- <table id="facet.attributes">
- <title>Facet attributes</title>
- <tgroup cols="2">
- <colspec colwidth="2*" colname="type"></colspec>
- <colspec colwidth="9*" colname="description"></colspec>
- <thead>
- <row>
- <entry>Type</entry>
- <entry>Description</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>1</entry>
- <entry>
- Field-name. This is often a string, eg "Author", "Year", etc.
- </entry>
- </row>
-
- <row>
- <entry>2</entry>
- <entry>
- Sort order. Value should be an integer.
- Value 0: count descending (frequency). Value 1: alpha ascending.
- </entry>
- </row>
-
- <row>
- <entry>3</entry>
- <entry>
- Number of terms requested.
- </entry>
- </row>
-
- <row>
- <entry>4</entry>
- <entry>
- Start offset.
- </entry>
- </row>
-
- </tbody>
- </tgroup>
- </table>
- </sect1>
- </chapter>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
+++ /dev/null
-<?xml version="1.0" standalone="no"?>
-<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
- "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
-[
- <!ENTITY % local SYSTEM "local.ent">
- %local;
- <!ENTITY % entities SYSTEM "entities.ent">
- %entities;
- <!ENTITY % idcommon SYSTEM "common/common.ent">
- %idcommon;
-]>
-<book>
- &bookinfo;
- &chap-introduction;
- &chap-installation;
- &chap-zoom;
- &chap-server;
- &chap-asn;
- &chap-soap;
- &chap-tools;
- &chap-odr;
- &chap-comstack;
- &chap-future;
-
- <reference id="reference">
- <title>Reference</title>
- <partintro id="reference-introduction">
- <para>
- The material in this chapter is drawn directly from the individual
- manual entries.
- </para>
- </partintro>
- &manref;
- </reference>
-
- <appendix id="list-oids"><title>List of Object Identifiers</title>
- <para>
- These is a list of object identifiers that are built into YAZ.
- </para>
- &std-oid-table;
- </appendix>
-
- <appendix id="bib1-diagnostics"><title>Bib-1 diagnostics</title>
- <para>
- List of Bib-1 diagnostics that are known to YAZ.
- </para>
- &bib1-diag-table;
- </appendix>
-
- <appendix id="sru-diagnostics"><title>SRU diagnostics</title>
- <para>
- List of SRU diagnostics that are known to YAZ.
- </para>
- &srw-diag-table;
- </appendix>
-
- &app-license;
- &app-indexdata;
- &app-credits;
-</book>
-
-<!-- Keep this comment at the end of the file
-Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
-End:
--->
+++ /dev/null
-<!--
-### Still to document:
-ZOOM_connection_errcode(c)
-ZOOM_connection_errmsg(c)
-ZOOM_connection_addinfo(c)
-ZOOM_connection_addinfo(c)
-ZOOM_connection_diagset(c);
-ZOOM_connection_save_apdu_wrbuf
-ZOOM_diag_str(error)
-ZOOM_resultset_record_immediate(s, pos)
-ZOOM_resultset_cache_reset(r)
-ZOOM_options_set_callback(opt, function, handle)
-ZOOM_options_create_with_parent2(parent1, parent2)
-ZOOM_options_getl(opt, name, len)
-ZOOM_options_setl(opt, name, value, len)
-ZOOM_options_get_bool(opt, name, defa)
-ZOOM_options_get_int(opt, name, defa)
-ZOOM_options_set_int(opt, name, value)
--->
- <chapter id="zoom"><title>ZOOM</title>
- <para>
- &zoom; is an acronym for 'Z39.50 Object-Orientation Model' and is
- an initiative started by Mike Taylor (Mike is from the UK, which
- explains the peculiar name of the model). The goal of &zoom; is to
- provide a common Z39.50 client API not bound to a particular
- programming language or toolkit.
- </para>
-
- <para>
- From YAZ version 2.1.12, <ulink url="&url.sru;">SRU</ulink> is supported.
- You can make SRU ZOOM connections by specifying scheme
- <literal>http://</literal> for the hostname for a connection.
- The dialect of SRU used is specified by the value of the
- connection's <literal>sru</literal> option, which may be SRU over
- HTTP GET (<literal>get</literal>),
- SRU over HTTP POST (<literal>post</literal>), (SRU over
- SOAP) (<literal>soap</literal>) or <literal>solr</literal>
- (<ulink url="&url.solr;">Solr</ulink> Web Service).
- Using the facility for embedding options in target strings, a
- connection can be forced to use SRU rather the SRW (the default) by
- prefixing the target string with <literal>sru=get,</literal>, like this:
- <literal>sru=get,http://sru.miketaylor.org.uk:80/sru.pl</literal>
- </para>
- <para>
- <ulink url="&url.solr;">Solr</ulink> protocol support was added to
- YAZ in version 4.1.0, as a dialect of a SRU protocol, since both are
- HTTP based protocols.
- </para>
- <para>
- The lack of a simple Z39.50 client API for &yaz; has become more
- and more apparent over time. So when the first &zoom; specification
- became available,
- an implementation for &yaz; was quickly developed. For the first time, it is
- now as easy (or easier!) to develop clients than servers with &yaz;. This
- chapter describes the &zoom; C binding. Before going further, please
- reconsider whether C is the right programming language for the job.
- There are other language bindings available for &yaz;, and still
- more
- are in active development. See the
- <ulink url="&url.zoom;">ZOOM web-site</ulink> for
- more information.
- </para>
-
- <para>
- In order to fully understand this chapter you should read and
- try the example programs <literal>zoomtst1.c</literal>,
- <literal>zoomtst2.c</literal>, .. in the <literal>zoom</literal>
- directory.
- </para>
-
- <para>
- The C language misses features found in object oriented languages
- such as C++, Java, etc. For example, you'll have to manually,
- destroy all objects you create, even though you may think of them as
- temporary. Most objects has a <literal>_create</literal> - and a
- <literal>_destroy</literal> variant.
- All objects are in fact pointers to internal stuff, but you don't see
- that because of typedefs. All destroy methods should gracefully ignore a
- <literal>NULL</literal> pointer.
- </para>
- <para>
- In each of the sections below you'll find a sub section called
- protocol behavior, that describes how the API maps to the Z39.50
- protocol.
- </para>
- <sect1 id="zoom-connections"><title>Connections</title>
-
- <para>The Connection object is a session with a target.
- </para>
- <synopsis>
- #include <yaz/zoom.h>
-
- ZOOM_connection ZOOM_connection_new(const char *host, int portnum);
-
- ZOOM_connection ZOOM_connection_create(ZOOM_options options);
-
- void ZOOM_connection_connect(ZOOM_connection c, const char *host,
- int portnum);
- void ZOOM_connection_destroy(ZOOM_connection c);
- </synopsis>
- <para>
- Connection objects are created with either function
- <function>ZOOM_connection_new</function> or
- <function>ZOOM_connection_create</function>.
- The former creates and automatically attempts to establish a network
- connection with the target. The latter doesn't establish
- a connection immediately, thus allowing you to specify options
- before establishing network connection using the function
- <function>ZOOM_connection_connect</function>.
- If the port number, <literal>portnum</literal>, is zero, the
- <literal>host</literal> is consulted for a port specification.
- If no port is given, 210 is used. A colon denotes the beginning of
- a port number in the host string. If the host string includes a
- slash, the following part specifies a database for the connection.
- </para>
- <para>
- You can prefix the host with a scheme followed by colon. The
- default scheme is <literal>tcp</literal> (Z39.50 protocol).
- The scheme <literal>http</literal> selects SRU/get over HTTP by default,
- but can overridded to use SRU/post, SRW and the Solr protocol.
- </para>
- <para>
- You can prefix the scheme-qualified host-string with one or more
- comma-separated
- <literal><parameter>key</parameter>=<parameter>value</parameter></literal>
- sequences, each of which represents an option to be set into the
- connection structure <emphasis>before</emphasis> the
- protocol-level connection is forged and the initialization
- handshake takes place. This facility can be used to provide
- authentication credentials, as in host-strings such as:
- <literal>user=admin,password=halfAm4n,tcp:localhost:8017/db</literal>
- </para>
- <para>
- Connection objects should be destroyed using the function
- <function>ZOOM_connection_destroy</function>.
- </para>
- <synopsis>
- void ZOOM_connection_option_set(ZOOM_connection c,
- const char *key, const char *val);
-
- void ZOOM_connection_option_setl(ZOOM_connection c,
- const char *key,
- const char *val, int len);
-
- const char *ZOOM_connection_option_get(ZOOM_connection c,
- const char *key);
- const char *ZOOM_connection_option_getl(ZOOM_connection c,
- const char *key,
- int *lenp);
- </synopsis>
- <para>
- The functions <function>ZOOM_connection_option_set</function> and
- <function>ZOOM_connection_option_setl</function> allows you to
- set an option given by <parameter>key</parameter> to the value
- <parameter>value</parameter> for the connection.
- For <function>ZOOM_connection_option_set</function>, the
- value is assumed to be a 0-terminated string. Function
- <function>ZOOM_connection_option_setl</function> specifies a
- value of a certain size (len).
- </para>
- <para>
- Functions <function>ZOOM_connection_option_get</function> and
- <function>ZOOM_connection_option_getl</function> returns
- the value for an option given by <parameter>key</parameter>.
- </para>
- <table id="zoom-connection-options" frame="top">
- <title>ZOOM Connection Options</title>
- <tgroup cols="3">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <colspec colwidth="3*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- implementationName</entry><entry>Name of Your client
- </entry><entry>none</entry></row>
- <row><entry>
- user</entry><entry>Authentication user name
- </entry><entry>none</entry></row>
- <row><entry>
- group</entry><entry>Authentication group name
- </entry><entry>none</entry></row>
- <row><entry>
- password</entry><entry>Authentication password.
- </entry><entry>none</entry></row>
- <row><entry>
- authenticationMode</entry><entry>How authentication is encoded.
- </entry><entry>basic</entry></row>
- <row><entry>
- host</entry><entry>Target host. This setting is "read-only".
- It's automatically set internally when connecting to a target.
- </entry><entry>none</entry></row>
- <row><entry>
- proxy</entry><entry>Proxy host. If set, the logical host
- is encoded in the otherInfo area of the Z39.50 Init PDU
- with OID 1.2.840.10003.10.1000.81.1.
- </entry><entry>none</entry></row>
- <row><entry>
- clientIP</entry><entry>Client IP. If set, is
- encoded in the otherInfo area of a Z39.50 PDU with OID
- 1.2.840.10003.10.1000.81.3. Holds the original IP addreses
- of a client. Is used of ZOOM is used in a gateway of some sort.
- </entry><entry>none</entry></row>
- <row><entry>
- async</entry><entry>If true (1) the connection operates in
- asynchronous operation which means that all calls are non-blocking
- except
- <link linkend="zoom.events"><function>ZOOM_event</function></link>.
- </entry><entry>0</entry></row>
- <row><entry>
- maximumRecordSize</entry><entry> Maximum size of single record.
- </entry><entry>1 MB</entry></row>
- <row><entry>
- preferredMessageSize</entry><entry> Maximum size of multiple records.
- </entry><entry>1 MB</entry></row>
- <row><entry>
- lang</entry><entry> Language for negotiation.
- </entry><entry>none</entry></row>
- <row><entry>
- charset</entry><entry> Character set for negotiation.
- </entry><entry>none</entry></row>
- <row><entry>
- serverImplementationId</entry><entry>
- Implementation ID of server. (The old targetImplementationId
- option is also supported for the benefit of old applications.)
- </entry><entry>none</entry></row>
- <row><entry>
- targetImplementationName</entry><entry>
- Implementation Name of server. (The old
- targetImplementationName option is also supported for the
- benefit of old applications.)
- </entry><entry>none</entry></row>
- <row><entry>
- serverImplementationVersion</entry><entry>
- Implementation Version of server. (the old
- targetImplementationVersion option is also supported for the
- benefit of old applications.)
- </entry><entry>none</entry></row>
- <row><entry>
- databaseName</entry><entry>One or more database names
- separated by character plus (<literal>+</literal>), which to
- be used by subsequent search requests on this Connection.
- </entry><entry>Default</entry></row>
- <row><entry>
- piggyback</entry><entry>True (1) if piggyback should be
- used in searches; false (0) if not.
- </entry><entry>1</entry></row>
- <row><entry>
- smallSetUpperBound</entry><entry>If hits is less than or equal to this
- value, then target will return all records using small element set name
- </entry><entry>0</entry></row>
- <row><entry>
- largeSetLowerBound</entry><entry>If hits is greater than this
- value, the target will return no records.
- </entry><entry>1</entry></row>
- <row><entry>
- mediumSetPresentNumber</entry><entry>This value represents
- the number of records to be returned as part of a search when when
- hits is less than or equal to large set lower bound and if hits
- is greater than small set upper bound.
- </entry><entry>0</entry></row>
- <row><entry>
- smallSetElementSetName</entry><entry>
- The element set name to be used for small result sets.
- </entry><entry>none</entry></row>
- <row><entry>
- mediumSetElementSetName</entry><entry>
- The element set name to be for medium-sized result sets.
- </entry><entry>none</entry></row>
- <row><entry>
- init_opt_search, init_opt_present, init_opt_delSet, etc.</entry><entry>
- After a successful Init, these options may be interrogated to
- discover whether the server claims to support the specified
- operations.
- </entry><entry>none</entry></row>
- <row>
- <entry>sru</entry><entry>
- SRU/Solr transport type. Must be either <literal>soap</literal>,
- <literal>get</literal>, <literal>post</literal>, or
- <literal>solr</literal>.
- </entry><entry>soap</entry></row>
- <row><entry>
- sru_version</entry><entry>
- SRU/SRW version. Should be <literal>1.1</literal>, or
- <literal>1.2</literal>. This is , prior to connect, the version
- to offer (highest version). And following connect (in fact
- first operation), holds the negotiated version with the server
- (same or lower version).
- </entry><entry>1.2</entry></row>
- <row id="zoom.facets.option"><entry>
- facets</entry><entry>
- Requested or recommend facets may be given before a search is sent.
- The value of this setting is described in <xref linkend="facets"/>
- For inspection of the facets returned, refer to the functions
- described in <xref linkend="zoom.facets"/>.
- </entry><entry>none</entry></row>
- <row><entry>
- apdulog</entry><entry>
- If set to a true value such as "1", a log of low-level
- protocol packets is emitted on standard error stream. This
- can be very useful for debugging.
- </entry><entry>0</entry></row>
- <row><entry>
- saveAPDU</entry><entry>
- If set to a true value such as "1", a log of low-level
- protocol packets is saved. The log can be retrieved by reading
- option APDU. Setting saveAPDU always has the side effect of
- resetting the currently saved log. This setting is
- <emphasis>write-only</emphasis>. If read, NULL will be returned.
- It is only recognized in
- <function>ZOOM_connection_option_set</function>.
- </entry><entry>0</entry></row>
- <row><entry>
- APDU</entry><entry>
- Returns the log of protocol packets. Will be empty if logging
- is not enabled (see saveAPDU above). This setting is
- <emphasis>read-only</emphasis>. It is only recognized if used
- in call to <function>ZOOM_connection_option_get</function> or
- <function>ZOOM_connection_option_getl</function>.
- </entry><entry></entry></row>
- <row><entry>
- memcached</entry><entry>
- If given and non-empty,
- <ulink url="&url.libmemcached;">libMemcached</ulink>
- will be configured for the connection.
- This option is inspected by ZOOM when a connection is established.
- If the <literal>memcached</literal> option is given
- and YAZ is compiled without libMemcached support, an internal
- diagnostic (10018) will be thrown.
- libMemcached support is available for YAZ 5.0.13 or later. If this
- option is supplied for an earlier version of YAZ, it is
- <emphasis>ignored</emphasis>.
- The value of this option is a string passed verbatim to
- the <function>memcached</function> function part of libMemcached.
- Refer to man page
- <ulink url="http://manned.org/memcached.3">memcached(3)</ulink>.
- Earlier versions of libMemcached
- do not offer this function. In this case only the option
- <literal>--server=</literal><replaceable>host</replaceable> may
- be given (YAZ emulates that part of libMemcached).
- </entry><entry>none</entry></row>
- </tbody>
- </tgroup>
- </table>
- <para>
- If either option <literal>lang</literal> or <literal>charset</literal>
- is set, then
- <ulink url="&url.z39.50.charneg;">
- Character Set and Language Negotiation</ulink> is in effect.
- </para>
- <synopsis>
- int ZOOM_connection_error(ZOOM_connection c, const char **cp,
- const char **addinfo);
- int ZOOM_connection_error_x(ZOOM_connection c, const char **cp,
- const char **addinfo, const char **dset);
- </synopsis>
- <para>
- Function <function>ZOOM_connection_error</function> checks for
- errors for the last operation(s) performed. The function returns
- zero if no errors occurred; non-zero otherwise indicating the error.
- Pointers <parameter>cp</parameter> and <parameter>addinfo</parameter>
- holds messages for the error and additional-info if passed as
- non-<literal>NULL</literal>. Function
- <function>ZOOM_connection_error_x</function> is an extended version
- of <function>ZOOM_connection_error</function> that is capable of
- returning name of diagnostic set in <parameter>dset</parameter>.
- </para>
- <sect2 id="zoom-connection-z39.50">
- <title>Z39.50 Protocol behavior</title>
- <para>
- The calls <function>ZOOM_connection_new</function> and
- <function>ZOOM_connection_connect</function> establishes a TCP/IP
- connection and sends an Initialize Request to the target if
- possible. In addition, the calls waits for an Initialize Response
- from the target and the result is inspected (OK or rejected).
- </para>
- <para>
- If <literal>proxy</literal> is set then the client will establish
- a TCP/IP connection with the peer as specified by the
- <literal>proxy</literal> host and the hostname as part of the
- connect calls will be set as part of the Initialize Request.
- The proxy server will then "forward" the PDU's transparently
- to the target behind the proxy.
- </para>
- <para>
- For the authentication parameters, if option <literal>user</literal>
- is set and both options <literal>group</literal> and
- <literal>pass</literal> are unset, then Open style
- authentication is used (Version 2/3) in which case the username
- is usually followed by a slash, then by a password.
- If either <literal>group</literal>
- or <literal>pass</literal> is set then idPass authentication
- (Version 3 only) is used. If none of the options are set, no
- authentication parameters are set as part of the Initialize Request
- (obviously).
- </para>
- <para>
- When option <literal>async</literal> is 1, it really means that
- all network operations are postponed (and queued) until the
- function <literal>ZOOM_event</literal> is invoked. When doing so
- it doesn't make sense to check for errors after
- <literal>ZOOM_connection_new</literal> is called since that
- operation "connecting - and init" is still incomplete and the
- API cannot tell the outcome (yet).
- </para>
- </sect2>
- <sect2 id="zoom.sru.init.behavior">
- <title>SRU/Solr Protocol behavior</title>
- <para>
- The HTTP based protocols (SRU, SRW, Solr) doesn't feature an
- Inititialize Request, so the connection phase merely establishes a
- TCP/IP connection with the HTTP server.
- </para>
- <para>Most of the ZOOM connection options do not
- affect SRU/Solr and they are ignored. However, future versions
- of &yaz; might honor <literal>implementationName</literal> and
- put that as part of User-Agent header for HTTP requests.
- </para>
- <para>
- The <literal>charset</literal> is used in the Content-Type header
- of HTTP requests.
- </para>
- <para>
- Setting <literal>authentcationMode</literal> specifies how
- authentication parameters are encoded for HTTP. The default is
- "<literal>basic</literal>" where <literal>user</literal> and
- <literal>password</literal> are encoded by using HTTP basic
- authentication.
- </para>
- <para>
- If <literal>authentcationMode</literal> is "<literal>url</literal>", then
- user and password are encoded in the URL by parameters
- <literal>x-username</literal> and <literal>x-password</literal> as
- given by the SRU standard.
- </para>
- </sect2>
- </sect1>
- <sect1 id="zoom.query"><title>Queries</title>
- <para>
- Query objects represents queries.
- </para>
- <synopsis>
- ZOOM_query ZOOM_query_create(void);
-
- void ZOOM_query_destroy(ZOOM_query q);
-
- int ZOOM_query_prefix(ZOOM_query q, const char *str);
-
- int ZOOM_query_cql(ZOOM_query s, const char *str);
-
- int ZOOM_query_sortby(ZOOM_query q, const char *criteria);
-
- int ZOOM_query_sortby2(ZOOM_query q, const char *strategy,
- const char *criteria);
- </synopsis>
- <para>
- Create query objects using <function>ZOOM_query_create</function>
- and destroy them by calling <function>ZOOM_query_destroy</function>.
- RPN-queries can be specified in <link linkend="PQF">PQF</link>
- notation by using the
- function <function>ZOOM_query_prefix</function>.
- The <function>ZOOM_query_cql</function> specifies a CQL
- query to be sent to the server/target.
- More query types will be added in future versions of &yaz;, such as
- <link linkend="CCL">CCL</link> to RPN-mapping, native CCL query,
- etc. In addition to a search, a sort criteria may be set. Function
- <function>ZOOM_query_sortby</function> enables Z39.50 sorting and
- it takes sort criteria using the same string notation as
- yaz-client's <link linkend="sortspec">sort command</link>.
- </para>
- <para id="zoom.query.sortby2">
- <function>ZOOM_query_sortby2</function> is similar to
- <function>ZOOM_query_sortby</function> but allows a strategy for
- sorting. The reason for the strategy parameter is that some
- protocols offers multiple ways of performing sorting.
- For example, Z39.50 has the standard sort, which is performed after
- search on an existing result set.
- It's also possible to use CQL in Z39.50 as the query type and use
- CQL's SORTBY keyword. Finally, Index Data's
- Zebra server also allows sorting to be specified as part of RPN (Type 7).
- </para>
- <table id="zoom-sort-strategy" frame="top">
- <title>ZOOM sort strategy</title>
- <tgroup cols="2">
- <colspec colwidth="2*" colname="name"/>
- <colspec colwidth="5*" colname="description"/>
- <thead>
- <row>
- <entry>Name</entry>
- <entry>Description</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>z39.50</entry><entry>Z39.50 resultset sort</entry>
- </row>
- <row>
- <entry>type7</entry><entry>Sorting embedded in RPN(Type-7)</entry>
- </row>
- <row>
- <entry>cql</entry><entry>CQL SORTBY</entry>
- </row>
- <row>
- <entry>sru11</entry><entry>SRU sortKeys parameter</entry>
- </row>
- <row>
- <entry>solr</entry><entry>Solr sort</entry>
- </row>
- <row>
- <entry>embed</entry><entry>type7 for Z39.50, cql for SRU,
- solr for Solr protocol</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
- </sect1>
- <sect1 id="zoom.resultsets"><title>Result sets</title>
- <para>
- The result set object is a container for records returned from
- a target.
- </para>
- <synopsis>
- ZOOM_resultset ZOOM_connection_search(ZOOM_connection, ZOOM_query q);
-
- ZOOM_resultset ZOOM_connection_search_pqf(ZOOM_connection c,
- const char *q);
- void ZOOM_resultset_destroy(ZOOM_resultset r);
- </synopsis>
- <para>
- Function <function>ZOOM_connection_search</function> creates
- a result set given a connection and query.
- Destroy a result set by calling
- <function>ZOOM_resultset_destroy</function>.
- Simple clients may using PQF only may use function
- <function>ZOOM_connection_search_pqf</function> in which case
- creating query objects is not necessary.
- </para>
- <synopsis>
- void ZOOM_resultset_option_set(ZOOM_resultset r,
- const char *key, const char *val);
-
- const char *ZOOM_resultset_option_get(ZOOM_resultset r, const char *key);
-
- size_t ZOOM_resultset_size(ZOOM_resultset r);
- </synopsis>
- <para>
- Functions <function>ZOOM_resultset_options_set</function> and
- <function>ZOOM_resultset_get</function> sets and gets an option
- for a result set similar to <function>ZOOM_connection_option_get</function>
- and <function>ZOOM_connection_option_set</function>.
- </para>
- <para>
- The number of hits also called result-count is returned by
- function <function>ZOOM_resultset_size</function>.
- </para>
- <table id="zoom.resultset.options"
- frame="top"><title>ZOOM Result set Options</title>
- <tgroup cols="3">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <colspec colwidth="2*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- start</entry><entry>Offset of first record to be
- retrieved from target. First record has offset 0 unlike the
- protocol specifications where first record has position 1.
- This option affects ZOOM_resultset_search and
- ZOOM_resultset_search_pqf and must be set before any of
- these functions are invoked. If a range of
- records must be fetched manually after search,
- function ZOOM_resultset_records should be used.
- </entry><entry>0</entry></row>
- <row><entry>
- count</entry><entry>Number of records to be retrieved.
- This option affects ZOOM_resultset_search and
- ZOOM_resultset_search_pqf and must be set before any of
- these functions are invoked.
- </entry><entry>0</entry></row>
- <row><entry>
- presentChunk</entry><entry>The number of records to be
- requested from the server in each chunk (present request). The
- value 0 means to request all the records in a single chunk.
- (The old <literal>step</literal>
- option is also supported for the benefit of old applications.)
- </entry><entry>0</entry></row>
- <row><entry>
- elementSetName</entry><entry>Element-Set name of records.
- Most targets should honor element set name <literal>B</literal>
- and <literal>F</literal> for brief and full respectively.
- </entry><entry>none</entry></row>
- <row><entry>
- preferredRecordSyntax</entry><entry>Preferred Syntax, such as
- <literal>USMARC</literal>, <literal>SUTRS</literal>, etc.
- </entry><entry>none</entry></row>
- <row><entry>
- schema</entry><entry>Schema for retrieval, such as
- <literal>Gils-schema</literal>, <literal>Geo-schema</literal>, etc.
- </entry><entry>none</entry></row>
- <row><entry>
- setname</entry><entry>Name of Result Set (Result Set ID).
- If this option isn't set, the ZOOM module will automatically
- allocate a result set name.
- </entry><entry>default</entry></row>
- <row><entry>
- rpnCharset</entry><entry>Character set for RPN terms.
- If this is set, ZOOM C will assume that the ZOOM application is
- running UTF-8. Terms in RPN queries are then converted to the
- rpnCharset. If this is unset, ZOOM C will not assume any encoding
- of RPN terms and no conversion is performed.
- </entry><entry>none</entry></row>
- </tbody>
- </tgroup>
- </table>
- <para>
- For servers that support Search Info report, the following
- options may be read using <function>ZOOM_resultset_get</function>.
- This detailed information is read after a successful search has
- completed.
- </para>
- <para>
- This information is a list of of items, where each item is
- information about a term or subquery. All items in the list
- are prefixed by
- <literal>SearchResult.</literal><replaceable>no</replaceable>
- where no presents the item number (0=first, 1=second).
- Read <literal>searchresult.size</literal> to determine the
- number of items.
- </para>
- <table id="zoom.search.info.report.options"
- frame="top"><title>Search Info Report Options</title>
- <tgroup cols="2">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>searchresult.size</entry>
- <entry>
- number of search result entries. This option is-nonexistant
- if no entries are returned by the server.
- </entry>
- </row>
- <row>
- <entry>searchresult.<replaceable>no</replaceable>.id</entry>
- <entry>sub query ID</entry>
- </row>
- <row>
- <entry>searchresult.<replaceable>no</replaceable>.count</entry>
- <entry>result count for item (number of hits)</entry>
- </row>
- <row>
- <entry>searchresult.<replaceable>no</replaceable>.subquery.term</entry>
- <entry>subquery term</entry>
- </row>
- <row>
- <entry>
- searchresult.<replaceable>no</replaceable>.interpretation.term
- </entry>
- <entry>interpretation term</entry>
- </row>
- <row>
- <entry>
- searchresult.<replaceable>no</replaceable>.recommendation.term
- </entry>
- <entry>recommendation term</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <sect2 id="zoom.z3950.resultset.sort">
- <title>Z39.50 Result-set Sort</title>
- <synopsis>
- void ZOOM_resultset_sort(ZOOM_resultset r,
- const char *sort_type, const char *sort_spec);
-
- int ZOOM_resultset_sort1(ZOOM_resultset r,
- const char *sort_type, const char *sort_spec);
- </synopsis>
- <para>
- <function>ZOOM_resultset_sort</function> and
- <function>ZOOM_resultset_sort1</function> both sort an existing
- result-set. The sort_type parameter is not use. Set it to "yaz".
- The sort_spec is same notation as ZOOM_query_sortby and identical
- to that offered by yaz-client's
- <link linkend="sortspec">sort command</link>.
- </para>
- <para>
- These functions only work for Z39.50. Use the more generic utility
- <link linkend="zoom.query.sortby2">
- <function>ZOOM_query_sortby2</function></link>
- for other protocols (and even Z39.50).
- </para>
- </sect2>
- <sect2 id="zoom.z3950.resultset.behavior">
- <title>Z39.50 Protocol behavior</title>
- <para>
- The creation of a result set involves at least a SearchRequest
- - SearchResponse protocol handshake. Following that, if a sort
- criteria was specified as part of the query, a SortRequest -
- SortResponse handshake takes place. Note that it is necessary to
- perform sorting before any retrieval takes place, so no records will
- be returned from the target as part of the SearchResponse because these
- would be unsorted. Hence, piggyback is disabled when sort criteria
- are set. Following Search - and a possible sort - Retrieval takes
- place - as one or more Present Requests/Response pairs being
- transferred.
- </para>
- <para>
- The API allows for two different modes for retrieval. A high level
- mode which is somewhat more powerful and a low level one.
- The low level is enabled when searching on a Connection object
- for which the settings
- <literal>smallSetUpperBound</literal>,
- <literal>mediumSetPresentNumber</literal> and
- <literal>largeSetLowerBound</literal> are set. The low level mode
- thus allows you to precisely set how records are returned as part
- of a search response as offered by the Z39.50 protocol.
- Since the client may be retrieving records as part of the
- search response, this mode doesn't work well if sorting is used.
- </para>
- <para>
- The high-level mode allows you to fetch a range of records from
- the result set with a given start offset. When you use this mode
- the client will automatically use piggyback if that is possible
- with the target and perform one or more present requests as needed.
- Even if the target returns fewer records as part of a present response
- because of a record size limit, etc. the client will repeat sending
- present requests. As an example, if option <literal>start</literal>
- is 0 (default) and <literal>count</literal> is 4, and
- <literal>piggyback</literal> is 1 (default) and no sorting criteria
- is specified, then the client will attempt to retrieve the 4
- records as part the search response (using piggyback). On the other
- hand, if either <literal>start</literal> is positive or if
- a sorting criteria is set, or if <literal>piggyback</literal>
- is 0, then the client will not perform piggyback but send Present
- Requests instead.
- </para>
- <para>
- If either of the options <literal>mediumSetElementSetName</literal> and
- <literal>smallSetElementSetName</literal> are unset, the value
- of option <literal>elementSetName</literal> is used for piggyback
- searches. This means that for the high-level mode you only have
- to specify one elementSetName option rather than three.
- </para>
- </sect2>
- <sect2 id="zoom.sru.resultset.behavior">
- <title>SRU Protocol behavior</title>
- <para>
- Current version of &yaz; does not take advantage of a result set id
- returned by the SRU server. Future versions might do, however.
- Since, the ZOOM driver does not save result set IDs any
- present (retrieval) is transformed to a SRU SearchRetrieveRequest
- with same query but, possibly, different offsets.
- </para>
- <para>
- Option <literal>schema</literal> specifies SRU schema
- for retrieval. However, options <literal>elementSetName</literal> and
- <literal>preferredRecordSyntax</literal> are ignored.
- </para>
- <para>
- Options <literal>start</literal> and <literal>count</literal>
- are supported by SRU.
- The remaining options
- <literal>piggyback</literal>,
- <literal>smallSetUpperBound</literal>,
- <literal>largeSetLowerBound</literal>,
- <literal>mediumSetPresentNumber</literal>,
- <literal>mediumSetElementSetName</literal>,
- <literal>smallSetElementSetName</literal> are
- unsupported.
- </para>
- <para>
- SRU supports CQL queries, <emphasis>not</emphasis> PQF.
- If PQF is used, however, the PQF query is transferred anyway
- using non-standard element <literal>pQuery</literal> in
- SRU SearchRetrieveRequest.
- </para>
- <para>
- Solr queries has to be done in Solr query format.
- </para>
- <para>
- Unfortunately, SRU or Solr does not define a database setting. Hence,
- <literal>databaseName</literal> is unsupported and ignored.
- However, the path part in host parameter for functions
- <function>ZOOM_connecton_new</function> and
- <function>ZOOM_connection_connect</function> acts as a
- database (at least for the &yaz; SRU server).
- </para>
- </sect2>
- </sect1>
- <sect1 id="zoom.records"><title>Records</title>
- <para>
- A record object is a retrieval record on the client side -
- created from result sets.
- </para>
- <synopsis>
- void ZOOM_resultset_records(ZOOM_resultset r,
- ZOOM_record *recs,
- size_t start, size_t count);
- ZOOM_record ZOOM_resultset_record(ZOOM_resultset s, size_t pos);
-
- const char *ZOOM_record_get(ZOOM_record rec, const char *type,
- size_t *len);
-
- int ZOOM_record_error(ZOOM_record rec, const char **msg,
- const char **addinfo, const char **diagset);
-
- ZOOM_record ZOOM_record_clone(ZOOM_record rec);
-
- void ZOOM_record_destroy(ZOOM_record rec);
- </synopsis>
- <para>
- References to temporary records are returned by functions
- <function>ZOOM_resultset_records</function> or
- <function>ZOOM_resultset_record</function>.
- </para>
- <para>
- If a persistent reference to a record is desired
- <function>ZOOM_record_clone</function> should be used.
- It returns a record reference that should be destroyed
- by a call to <function>ZOOM_record_destroy</function>.
- </para>
- <para>
- A single record is returned by function
- <function>ZOOM_resultset_record</function> that takes a
- position as argument. First record has position zero.
- If no record could be obtained <literal>NULL</literal> is returned.
- </para>
- <para>
- Error information for a record can be checked with
- <function>ZOOM_record_error</function> which returns non-zero
- (error code) if record is in error, called <emphasis>Surrogate
- Diagnostics</emphasis> in Z39.50.
- </para>
- <para>
- Function <function>ZOOM_resultset_records</function> retrieves
- a number of records from a result set. Parameter <literal>start</literal>
- and <literal>count</literal> specifies the range of records to
- be returned. Upon completion array
- <literal>recs[0], ..recs[count-1]</literal>
- holds record objects for the records. The array of records
- <literal>recs</literal> should be allocated prior the call
- <function>ZOOM_resultset_records</function>. Note that for those
- records that couldn't be retrieved from the target
- <literal>recs[ ..]</literal> is set to <literal>NULL</literal>.
- </para>
- <para id="zoom.record.get">
- In order to extract information about a single record,
- <function>ZOOM_record_get</function> is provided. The
- function returns a pointer to certain record information. The
- nature (type) of the pointer depends on the parameter,
- <parameter>type</parameter>.
- </para>
- <para>
- The <parameter>type</parameter> is a string of the format:
- </para>
- <para>
- <replaceable>format</replaceable>[;charset=<replaceable>from</replaceable>[/<replaceable>opacfrom</replaceable>][,<replaceable>to</replaceable>]][;format=<replaceable>v</replaceable>]
- </para>
- <para>
- where <replaceable>format</replaceable> specifies the format of the
- returned record, <replaceable>from</replaceable>
- specifies the character set of the record in its original form
- (as returned by the server), <replaceable>to</replaceable> specifies
- the output (returned)
- character set encoding.
- If <replaceable>to</replaceable> is omitted UTF-8 is assumed.
- If charset is not given, then no character set conversion takes place.
- </para>
-
- <para>OPAC records may be returned in a different
- set from the bibliographic MARC record. If this is this the case,
- <replaceable>opacfrom</replaceable> should be set to the character set
- of the OPAC record part.
- </para>
- <note>
- <para>
- Specifying the OPAC record character set requires YAZ 4.1.5 or later.
- </para>
- </note>
- <para>
- The format argument controls whether record data should be XML
- pretty-printed (post process operation).
- It is enabled only if format value <replaceable>v</replaceable> is
- <literal>1</literal> and the record content is XML well-formed.
- </para>
- <para>
- In addition, for certain types, the length
- <literal>len</literal> passed will be set to the size in bytes of
- the returned information.
- </para>
- <para>
- The following are the supported values for <replaceable>form</replaceable>.
- <variablelist>
- <varlistentry><term><literal>database</literal></term>
- <listitem><para>Database of record is returned
- as a C null-terminated string. Return type
- <literal>const char *</literal>.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>syntax</literal></term>
- <listitem><para>The transfer syntax of the record is returned
- as a C null-terminated string containing the symbolic name of
- the record syntax, e.g. <literal>Usmarc</literal>. Return type
- is
- <literal>const char *</literal>.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>schema</literal></term>
- <listitem><para>The schema of the record is returned
- as a C null-terminated string. Return type is
- <literal>const char *</literal>.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>render</literal></term>
- <listitem><para>The record is returned in a display friendly
- format. Upon completion buffer is returned
- (type <literal>const char *</literal>) and length is stored in
- <literal>*len</literal>.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>raw</literal></term>
- <listitem><para>The record is returned in the internal
- YAZ specific format. For GRS-1, Explain, and others, the
- raw data is returned as type
- <literal>Z_External *</literal> which is just the type for
- the member <literal>retrievalRecord</literal> in
- type <literal>NamePlusRecord</literal>.
- For SUTRS and octet aligned record (including all MARCs) the
- octet buffer is returned and the length of the buffer.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>xml</literal></term>
- <listitem><para>The record is returned in XML if possible.
- SRU, Solr and Z39.50 records with transfer syntax XML are
- returned verbatim. MARC records are returned in
- <ulink url="&url.marcxml;">
- MARCXML
- </ulink>
- (converted from ISO2709 to MARCXML by YAZ).
- OPAC records are also converted to XML and the
- bibliographic record is converted to MARCXML (when possible).
- GRS-1 records are not supported for this form.
- Upon completion, the XML buffer is returned
- (type <literal>const char *</literal>) and length is stored in
- <literal>*len</literal>.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>opac</literal></term>
- <listitem><para>OPAC information for record is returned in XML
- if an OPAC record is present at the position given. If no
- OPAC record is present, a NULL pointer is returned.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>txml</literal></term>
- <listitem><para>The record is returned in TurboMARC if possible.
- SRU and Z39.50 records with transfer syntax XML are
- returned verbatim. MARC records are returned in
- <link linkend="tools.turbomarc">
- TurboMARC
- </link>
- (converted from ISO2709 to TurboMARC by YAZ).
- Upon completion, the XML buffer is returned
- (type <literal>const char *</literal>) and length is stored in
- <literal>*len</literal>.
- </para></listitem>
- </varlistentry>
- <varlistentry><term><literal>json</literal></term>
- <listitem><para>Like xml, but MARC records are converted to
- <ulink url="&url.marc_in_json;">MARC-in-JSON</ulink>.
- </para></listitem>
- </varlistentry>
-
- </variablelist>
- </para>
- <para>
- Most
- <ulink url="&url.marc21;">MARC21</ulink>
- records uses the
- <ulink url="&url.marc8;">MARC-8</ulink>
- character set encoding.
- An application that wishes to display in Latin-1 would use
- <screen>
- render; charset=marc8,iso-8859-1
- </screen>
- </para>
- <sect2 id="zoom.z3950.record.behavior">
- <title>Z39.50 Protocol behavior</title>
- <para>
- The functions <function>ZOOM_resultset_record</function> and
- <function>ZOOM_resultset_records</function> inspects the client-side
- record cache. Records not found in cache are fetched using
- Present.
- The functions may block (and perform network I/O) - even though option
- <literal>async</literal> is 1, because they return records objects.
- (and there's no way to return records objects without retrieving them!).
- </para>
- <para>
- There is a trick, however, in the usage of function
- <function>ZOOM_resultset_records</function> that allows for
- delayed retrieval (and makes it non-blocking). By using
- a null pointer for <parameter>recs</parameter> you're indicating
- you're not interested in getting records objects
- <emphasis>now</emphasis>.
- </para>
- </sect2>
- <sect2 id="zoom.sru.record.behavior">
- <title>SRU/Solr Protocol behavior</title>
- <para>
- The ZOOM driver for SRU/Solr treats records returned by a SRU/Solr server
- as if they where Z39.50 records with transfer syntax XML and
- no element set name or database name.
- </para>
- </sect2>
- </sect1>
- <sect1 id="zoom.facets"><title>Facets</title>
- <para>
- Facet operations is not part of the official ZOOM specification, but
- is an Index Data extension for YAZ-based Z39.50 targets,
- <ulink url="&url.solr;">Solr</ulink> and SRU 2.0 targets.
-
- Facets may be requestd by the
- <link linkend="zoom.facets.option">facets</link> option before a
- search is sent.
- For inspection of the returned facets, the following functions are
- available:
- </para>
- <synopsis>
- ZOOM_facet_field *ZOOM_resultset_facets(ZOOM_resultset r);
-
- ZOOM_facet_field ZOOM_resultset_get_facet_field(ZOOM_resultset r,
- const char *facet_name);
-
- ZOOM_facet_field ZOOM_resultset_get_facet_field_by_index(ZOOM_resultset r,
- int pos);
-
- size_t ZOOM_resultset_facets_size(ZOOM_resultset r);
-
- const char *ZOOM_facet_field_name(ZOOM_facet_field facet_field);
-
- size_t ZOOM_facet_field_term_count(ZOOM_facet_field facet_field);
-
- const char *ZOOM_facet_field_get_term(ZOOM_facet_field facet_field,
- size_t idx, int *freq);
- </synopsis>
- <para>
- References to temporary structures are returned by all functions.
- They are only valid as long the Result set is valid.
- <function>ZOOM_resultset_get_facet_field</function> or
- <function>ZOOM_resultset_get_facet_field_by_index</function>.
- <function>ZOOM_resultset_facets</function>.
- <function>ZOOM_facet_field_name</function>.
- <function>ZOOM_facet_field_get_term</function>.
- </para>
- <para id="zoom.resultset.get_facet_field">
- A single Facet field is returned by function
- <function>ZOOM_resultset_get_facet_field</function> or
- <function>ZOOM_resultset_get_facet_field_by_index</function> that takes
- a result set and facet name or positive index respectively. First
- facet has position zero. If no facet could be obtained (invalid name
- or index out of bounds) <literal>NULL</literal> is returned.
- </para>
- <para id="zoom.resultset.facets">
- An array of facets field can be returned by
- <function>ZOOM_resultset_facets</function>. The length of the array is
- given by <function>ZOOM_resultset_facets_size</function>. The array is
- zero-based and last entry will be at
- <function>ZOOM_resultset_facets_size(result_set)</function>-1.
- </para>
- <para id="zoom.resultset.facets_names">
- It is possible to interate over facets by name, by calling
- <function>ZOOM_resultset_facets_names</function>.
- This will return an const array of char * where each string can be used
- as parameter for <function>ZOOM_resultset_get_facet_field</function>.
- </para>
- <para>
- Function <function>ZOOM_facet_field_name</function> gets the request
- facet name from a returned facet field.
- </para>
- <para>
- Function <function>ZOOM_facet_field_get_term</function> returns the
- idx'th term and term count for a facet field.
- Idx must between 0 and
- <function>ZOOM_facet_field_term_count</function>-1, otherwise the
- returned reference will be <literal>NULL</literal>. On a valid idx, the
- value of the freq reference will be the term count.
- The <literal>freq</literal> parameter must be valid pointer to integer.
- </para>
- </sect1>
- <sect1 id="zoom.scan"><title>Scan</title>
- <para>
- This section describes an interface for Scan. Scan is not an
- official part of the ZOOM model yet. The result of a scan operation
- is the <literal>ZOOM_scanset</literal> which is a set of terms
- returned by a target.
- </para>
-
- <para>
- The Scan interface is supported for both Z39.50, SRU and Solr.
- </para>
-
- <synopsis>
- ZOOM_scanset ZOOM_connection_scan(ZOOM_connection c,
- const char *startpqf);
-
- ZOOM_scanset ZOOM_connection_scan1(ZOOM_connection c,
- ZOOM_query q);
-
- size_t ZOOM_scanset_size(ZOOM_scanset scan);
-
- const char *ZOOM_scanset_term(ZOOM_scanset scan, size_t pos,
- size_t *occ, size_t *len);
-
- const char *ZOOM_scanset_display_term(ZOOM_scanset scan, size_t pos,
- size_t *occ, size_t *len);
-
- void ZOOM_scanset_destroy(ZOOM_scanset scan);
-
- const char *ZOOM_scanset_option_get(ZOOM_scanset scan,
- const char *key);
-
- void ZOOM_scanset_option_set(ZOOM_scanset scan, const char *key,
- const char *val);
- </synopsis>
- <para>
- The scan set is created by function
- <function>ZOOM_connection_scan</function> which performs a scan
- operation on the connection using the specified
- <parameter>startpqf</parameter>.
- If the operation was successful, the size of the scan set can be
- retrieved by a call to <function>ZOOM_scanset_size</function>.
- Like result sets, the items are numbered 0,..size-1.
- To obtain information about a particular scan term, call function
- <function>ZOOM_scanset_term</function>. This function takes
- a scan set offset <literal>pos</literal> and returns a pointer
- to a <emphasis>raw term</emphasis> or <literal>NULL</literal> if
- non-present.
- If present, the <literal>occ</literal> and <literal>len</literal>
- are set to the number of occurrences and the length
- of the actual term respectively.
- <function>ZOOM_scanset_display_term</function> is similar to
- <function>ZOOM_scanset_term</function> except that it returns
- the <emphasis>display term</emphasis> rather than the raw term.
- In a few cases, the term is different from display term. Always
- use the display term for display and the raw term for subsequent
- scan operations (to get more terms, next scan result, etc).
- </para>
- <para>
- A scan set may be freed by a call to function
- <function>ZOOM_scanset_destroy</function>.
- Functions <function>ZOOM_scanset_option_get</function> and
- <function>ZOOM_scanset_option_set</function> retrieves and sets
- an option respectively.
- </para>
-
- <para>
- The <parameter>startpqf</parameter> is a subset of PQF, namely
- the Attributes+Term part. Multiple <literal>@attr</literal> can
- be used. For example to scan in title (complete) phrases:
- <literallayout>
- @attr 1=4 @attr 6=2 "science o"
- </literallayout>
- </para>
-
- <para>
- The <function>ZOOM_connecton_scan1</function> is a newer and
- more generic alternative to <function>ZOOM_connection_scan</function>
- which allows to use both CQL and PQF for Scan.
- </para>
-
- <table frame="top" id="zoom.scanset.options">
- <title>ZOOM Scan Set Options</title>
- <tgroup cols="3">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <colspec colwidth="2*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row><entry>
- number</entry><entry>Number of Scan Terms requested in next scan.
- After scan it holds the actual number of terms returned.
- </entry><entry>20</entry></row>
- <row><entry>
- position</entry><entry>Preferred Position of term in response
- in next scan; actual position after completion of scan.
- </entry><entry>1</entry></row>
- <row><entry>
- stepSize</entry><entry>Step Size
- </entry><entry>0</entry></row>
- <row><entry>
- scanStatus</entry><entry>An integer indicating the Scan Status
- of last scan.
- </entry><entry>0</entry></row>
- <row><entry>
- rpnCharset</entry><entry>Character set for RPN terms.
- If this is set, ZOOM C will assume that the ZOOM application is
- running UTF-8. Terms in RPN queries are then converted to the
- rpnCharset. If this is unset, ZOOM C will not assume any encoding
- of RPN terms and no conversion is performed.
- </entry><entry>none</entry></row>
- </tbody>
- </tgroup>
- </table>
- </sect1>
-
- <sect1 id="zoom.extendedservices"><title>Extended Services</title>
- <para>
- ZOOM offers an interface to a subset of the Z39.50 extended services
- as well as a few privately defined ones:
- </para>
- <itemizedlist>
- <listitem>
- <para>
- Z39.50 Item Order (ILL).
- See <xref linkend="zoom.item.order"/>.
- </para>
- </listitem>
- <listitem>
- <para>
- Record Update. This allows a client to insert, modify or delete
- records.
- See <xref linkend="zoom.record.update"/>.
- </para>
- </listitem>
- <listitem>
- <para>
- Database Create. This a non-standard feature. Allows a client
- to create a database.
- See <xref linkend="zoom.database.create"/>.
- </para>
- </listitem>
- <listitem>
- <para>
- Database Drop. This a non-standard feature. Allows a client
- to delete/drop a database.
- See <xref linkend="zoom.database.drop"/>.
- </para>
- </listitem>
- <listitem>
- <para>
- Commit operation. This a non-standard feature. Allows a client
- to commit operations.
- See <xref linkend="zoom.commit"/>.
- </para>
- </listitem>
- <!-- all the ILL PDU options should go here too -->
- </itemizedlist>
- <para>
- To create an extended service operation a <literal>ZOOM_package</literal>
- must be created. The operation is a five step operation. The
- package is created, package is configured by means of options,
- the package is send, result is inspected (by means of options),
- the package is destroyed.
- </para>
- <synopsis>
- ZOOM_package ZOOM_connection_package(ZOOM_connection c,
- ZOOM_options options);
-
- const char *ZOOM_package_option_get(ZOOM_package p,
- const char *key);
- void ZOOM_package_option_set(ZOOM_package p, const char *key,
- const char *val);
- void ZOOM_package_send(ZOOM_package p, const char *type);
-
- void ZOOM_package_destroy(ZOOM_package p);
- </synopsis>
- <para>
- The <function>ZOOM_connection_package</function> creates a
- package for the connection given using the options specified.
- </para>
- <para>
- Functions <function>ZOOM_package_option_get</function> and
- <function>ZOOM_package_option_set</function> gets and sets
- options.
- </para>
- <para>
- <function>ZOOM_package_send</function> sends
- the package the via connection specified in
- <function>ZOOM_connection_package</function>.
- The <parameter>type</parameter> specifies the actual extended service
- package type to be sent.
- </para>
-
- <table frame="top" id="zoom.extendedservices.options">
- <title>Extended Service Common Options</title>
- <tgroup cols="3">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <colspec colwidth="3*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>package-name</entry>
- <entry>Extended Service Request package name. Must be specified
- as part of a request</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>user-id</entry>
- <entry>User ID of Extended Service Package. Is a request option</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>function</entry>
- <entry>
- Function of package - one of <literal>create</literal>,
- <literal>delete</literal>, <literal>modify</literal>. Is
- a request option.
- </entry>
- <entry><literal>create</literal></entry>
- </row>
- <row>
- <entry>waitAction</entry>
- <entry>
- Wait action for package. Possible values:
- <literal>wait</literal>, <literal>waitIfPossible</literal>,
- <literal>dontWait</literal> or <literal>dontReturnPackage</literal>.
- </entry>
- <entry><literal>waitIfPossible</literal></entry>
- </row>
- <row>
- <entry>targetReference</entry>
- <entry>
- Target Reference. This is part of the response as returned
- by the server. Read it after a successful operation.
- </entry>
- <entry><literal>none</literal></entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- <sect2 id="zoom.item.order"><title>Item Order</title>
- <para>
- For Item Order, type must be set to <literal>itemorder</literal> in
- <function>ZOOM_package_send</function>.
- </para>
-
- <table frame="top" id="zoom.item.order.options">
- <title>Item Order Options</title>
- <tgroup cols="3">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <colspec colwidth="3*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>contact-name</entry>
- <entry>ILL contact name</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>contact-phone</entry>
- <entry>ILL contact phone</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>contact-email</entry>
- <entry>ILL contact email</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>itemorder-item</entry>
- <entry>Position for item (record) requested. An integer</entry>
- <entry>1</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- </sect2>
-
- <sect2 id="zoom.record.update"><title>Record Update</title>
- <para>
- For Record Update, type must be set to <literal>update</literal> in
- <function>ZOOM_package_send</function>.
- </para>
-
- <table frame="top" id="zoom.record.update.options">
- <title>Record Update Options</title>
- <tgroup cols="3">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <colspec colwidth="3*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>action</entry>
- <entry>
- The update action. One of
- <literal>specialUpdate</literal>,
- <literal>recordInsert</literal>,
- <literal>recordReplace</literal>,
- <literal>recordDelete</literal>,
- <literal>elementUpdate</literal>.
- </entry>
- <entry><literal>specialUpdate (recordInsert for updateVersion=1 which does not support specialUpdate)</literal></entry>
- </row>
- <row>
- <entry>recordIdOpaque</entry>
- <entry>Opaque Record ID</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>recordIdNumber</entry>
- <entry>Record ID number</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>record</entry>
- <entry>The record itself</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>recordOpaque</entry>
- <entry>Specifies an opaque record which is
- encoded as an ASN.1 ANY type with the OID as tiven by option
- <literal>syntax</literal> (see below).
- Option <literal>recordOpaque</literal> is an alternative
- to record - and <literal>record</literal> option (above) is
- ignored if recordOpaque is set. This option is only available in
- YAZ 3.0.35 and later and is meant to facilitate Updates with
- servers from OCLC.
- </entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>syntax</entry>
- <entry>The record syntax (transfer syntax). Is a string that
- is a known record syntax.
- </entry>
- <entry>no syntax</entry>
- </row>
- <row>
- <entry>databaseName</entry>
- <entry>Database from connection object</entry>
- <entry>Default</entry>
- </row>
- <row>
- <entry>correlationInfo.note</entry>
- <entry>Correlation Info Note (string)</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>correlationInfo.id</entry>
- <entry>Correlation Info ID (integer)</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>elementSetName</entry>
- <entry>Element Set for Record</entry>
- <entry>none</entry>
- </row>
- <row>
- <entry>updateVersion</entry>
- <entry>Record Update version which holds one of the values
- 1, 2 or 3. Each version has a distinct OID:
- 1.2.840.10003.9.5
- (<ulink url="&url.z39.50.extupdate1;">first version</ulink>) ,
- 1.2.840.10003.9.5.1
- (second version) and
- 1.2.840.10003.9.5.1.1
- (<ulink url="&url.z39.50.extupdate3;">third and
- newest version</ulink>).
- </entry>
- <entry>3</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
-
- </sect2>
-
- <sect2 id="zoom.database.create"><title>Database Create</title>
- <para>
- For Database Create, type must be set to <literal>create</literal> in
- <function>ZOOM_package_send</function>.
- </para>
-
- <table frame="top" id="zoom.database.create.options">
- <title>Database Create Options</title>
- <tgroup cols="3">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <colspec colwidth="3*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>databaseName</entry>
- <entry>Database from connection object</entry>
- <entry>Default</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
- </sect2>
-
- <sect2 id="zoom.database.drop"><title>Database Drop</title>
- <para>
- For Database Drop, type must be set to <literal>drop</literal> in
- <function>ZOOM_package_send</function>.
- </para>
-
- <table frame="top" id="zoom.database.drop.options">
- <title>Database Drop Options</title>
- <tgroup cols="3">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <colspec colwidth="3*" colname="default"></colspec>
- <thead>
- <row>
- <entry>Option</entry>
- <entry>Description</entry>
- <entry>Default</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>databaseName</entry>
- <entry>Database from connection object</entry>
- <entry>Default</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
- </sect2>
-
- <sect2 id="zoom.commit"><title>Commit Operation</title>
- <para>
- For Commit, type must be set to <literal>commit</literal> in
- <function>ZOOM_package_send</function>.
- </para>
- </sect2>
-
- <sect2 id="zoom.extended.services.behavior">
- <title>Protocol behavior</title>
- <para>
- All the extended services are Z39.50-only.
- </para>
- <note>
- <para>
- The database create, drop and commit services are privately defined
- operations.
- Refer to <filename>esadmin.asn</filename> in YAZ for the ASN.1
- definitions.
- </para>
- </note>
- </sect2>
- </sect1>
-
- <sect1 id="zoom.options"><title>Options</title>
- <para>
- Most &zoom; objects provide a way to specify options to change behavior.
- From an implementation point of view a set of options is just like
- an associative array / hash.
- </para>
- <synopsis>
- ZOOM_options ZOOM_options_create(void);
-
- ZOOM_options ZOOM_options_create_with_parent(ZOOM_options parent);
-
- void ZOOM_options_destroy(ZOOM_options opt);
- </synopsis>
- <synopsis>
- const char *ZOOM_options_get(ZOOM_options opt, const char *name);
-
- void ZOOM_options_set(ZOOM_options opt, const char *name,
- const char *v);
- </synopsis>
- <synopsis>
- typedef const char *(*ZOOM_options_callback)
- (void *handle, const char *name);
-
- ZOOM_options_callback
- ZOOM_options_set_callback(ZOOM_options opt,
- ZOOM_options_callback c,
- void *handle);
- </synopsis>
- </sect1>
-
- <sect1 id="zoom.queryconversions"><title>Query conversions</title>
- <synopsis>
- int ZOOM_query_cql2rpn(ZOOM_query s, const char *cql_str,
- ZOOM_connection conn);
-
- int ZOOM_query_ccl2rpn(ZOOM_query s, const char *ccl_str,
- const char *config,
- int *ccl_error, const char **error_string,
- int *error_pos);
- </synopsis>
- <para>
- <function>ZOOM_query_cql2rpn</function> translates the CQL string,
- client-side, into RPN which may be passed to the server.
- This is useful for server's that don't themselves
- support CQL, for which <function>ZOOM_query_cql</function> is useless.
- `conn' is used only as a place to stash diagnostics if compilation
- fails; if this information is not needed, a null pointer may be used.
- The CQL conversion is driven by option <literal>cqlfile</literal> from
- connection conn. This specifies a conversion file (eg pqf.properties)
- which <emphasis>must</emphasis> be present.
- </para>
- <para>
- <function>ZOOM_query_ccl2rpn</function> translates the CCL string,
- client-side, into RPN which may be passed to the server.
- The conversion is driven by the specification given by
- <literal>config</literal>. Upon completion 0 is returned on success; -1
- is returned on on failure. Om failure <literal>error_string</literal> and
- <literal>error_pos</literal> holds error message and position of
- first error in original CCL string.
- </para>
- </sect1>
- <sect1 id="zoom.events"><title>Events</title>
- <para>
- If you're developing non-blocking applications, you have to deal
- with events.
- </para>
- <synopsis>
- int ZOOM_event(int no, ZOOM_connection *cs);
- </synopsis>
- <para>
- The <function>ZOOM_event</function> executes pending events for
- a number of connections. Supply the number of connections in
- <literal>no</literal> and an array of connections in
- <literal>cs</literal> (<literal>cs[0] ... cs[no-1]</literal>).
- A pending event could be a sending a search, receiving a response,
- etc.
- When an event has occurred for one of the connections, this function
- returns a positive integer <literal>n</literal> denoting that an event
- occurred for connection <literal>cs[n-1]</literal>.
- When no events are pending for the connections, a value of zero is
- returned.
- To ensure that all outstanding requests are performed call this function
- repeatedly until zero is returned.
- </para>
- <para>
- If <function>ZOOM_event</function> returns and returns non-zero, the
- last event that occurred can be expected.
- </para>
- <synopsis>
- int ZOOM_connection_last_event(ZOOM_connection cs);
- </synopsis>
- <para>
- <function>ZOOM_connection_last_event</function> returns an event type
- (integer) for the last event.
- </para>
-
- <table frame="top" id="zoom.event.ids">
- <title>ZOOM Event IDs</title>
- <tgroup cols="2">
- <colspec colwidth="4*" colname="name"></colspec>
- <colspec colwidth="7*" colname="description"></colspec>
- <thead>
- <row>
- <entry>Event</entry>
- <entry>Description</entry>
- </row>
- </thead>
- <tbody>
- <row>
- <entry>ZOOM_EVENT_NONE</entry>
- <entry>No event has occurred</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_CONNECT</entry>
- <entry>TCP/IP connect has initiated</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_SEND_DATA</entry>
- <entry>Data has been transmitted (sending)</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_RECV_DATA</entry>
- <entry>Data has been received)</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_TIMEOUT</entry>
- <entry>Timeout</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_UNKNOWN</entry>
- <entry>Unknown event</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_SEND_APDU</entry>
- <entry>An APDU has been transmitted (sending)</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_RECV_APDU</entry>
- <entry>An APDU has been received</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_RECV_RECORD</entry>
- <entry>A result-set record has been received</entry>
- </row>
- <row>
- <entry>ZOOM_EVENT_RECV_SEARCH</entry>
- <entry>A search result been received</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
- </sect1>
- </chapter>
-
- <!-- Keep this comment at the end of the file
- Local variables:
- mode: sgml
- sgml-omittag:t
- sgml-shorttag:t
- sgml-minimize-attributes:nil
- sgml-always-quote-attributes:t
- sgml-indent-step:1
- sgml-indent-data:t
- sgml-parent-document: "yaz.xml"
- sgml-local-catalogs: nil
- sgml-namecase-general:t
- End:
- -->
-
<!-- Keep this comment at the end of the file
Local variables:
-mode: sgml
-sgml-omittag:t
-sgml-shorttag:t
-sgml-minimize-attributes:nil
-sgml-always-quote-attributes:t
-sgml-indent-step:1
-sgml-indent-data:t
-sgml-parent-document:nil
-sgml-local-catalogs: nil
-sgml-namecase-general:t
+mode: nxml
+nxml-child-indent: 1
End:
-->
projects / yaz-moved-to-github.git / commitdiff