基于OAIPMH協(xié)議數(shù)字圖書(shū)館中數(shù)據(jù)提供者的研究

1、隨著網(wǎng)絡(luò)、計(jì)算機(jī)和通信技術(shù)的快速發(fā)展，數(shù)字圖書(shū)館已經(jīng)成為人們?nèi)〉眯畔⒌闹匾獊?lái)源，然而對(duì)于網(wǎng)絡(luò)上眾多的分布式數(shù)字資源1?2，人們希望通過(guò)檢索獲得所需的信息。但由于早期在建立數(shù)字圖書(shū)館時(shí)沒(méi)有統(tǒng)一的標(biāo)準(zhǔn)，其資源的元數(shù)據(jù)格式并不相同，因而要制定出一套能夠描述不同數(shù)字圖書(shū)館資源的統(tǒng)一元數(shù)據(jù)格式相當(dāng)困難；同時(shí)，早期建立的數(shù)字圖書(shū)館數(shù)據(jù)提供者與服務(wù)提供者間多是點(diǎn)對(duì)點(diǎn)的協(xié)議服務(wù)，協(xié)議外的數(shù)據(jù)提供者與服務(wù)提供者間無(wú)法進(jìn)行元數(shù)據(jù)交互3?4。針對(duì)上述問(wèn)題，設(shè)計(jì)了數(shù)字圖書(shū)館中廣泛使用的元數(shù)據(jù)格式MARC和DC的格式轉(zhuǎn)換器，并重新設(shè)計(jì)了基于OAI?PMH協(xié)議互操作平臺(tái)的命令動(dòng)詞，對(duì)數(shù)字圖書(shū)館間資源的共享具有重要意義。1

2、 OAI?PMH協(xié)議概述數(shù)據(jù)提供者、服務(wù)提供者、注冊(cè)服務(wù)器構(gòu)成了數(shù)字圖書(shū)館領(lǐng)域5?7中OAI?PMH協(xié)議的框架。數(shù)據(jù)提供者是框架中存儲(chǔ)大量元數(shù)據(jù)的信息庫(kù)，服務(wù)提供者可從多個(gè)數(shù)據(jù)提供者中提取元數(shù)據(jù)并提供檢索、瀏覽等增值服務(wù)，服務(wù)提供者對(duì)數(shù)據(jù)提供者發(fā)出的是到UDP的請(qǐng)求，也就是基于OAI?PMH的命令動(dòng)詞，數(shù)據(jù)提供者對(duì)服務(wù)提供者的請(qǐng)求是通過(guò)HTTP協(xié)議以XML的形式響應(yīng)的，注冊(cè)服務(wù)器中可以進(jìn)行數(shù)據(jù)提供者和服務(wù)提供者的注冊(cè)。其具體框架如圖1所示。2 MARC到DC格式轉(zhuǎn)換器的設(shè)計(jì)與實(shí)現(xiàn)目前數(shù)字圖書(shū)管中使用的元數(shù)據(jù)格式主要為MARC，而基于OAI?PMH協(xié)議的數(shù)據(jù)提供者只能支持DC格式的元數(shù)據(jù)8。所以

3、要實(shí)現(xiàn)基于OAI?PMH協(xié)議的元數(shù)據(jù)互操作平臺(tái)，就需要把MARC格式的元數(shù)據(jù)轉(zhuǎn)換為DC格式的元數(shù)據(jù)。2.1 MARC與DC格式的域?qū)φ贞P(guān)系DC格式包含的元素描述共有15個(gè)，按照其描述內(nèi)容的范圍和類(lèi)別可分為三組，具體如表1所示。本系統(tǒng)提供了一個(gè)項(xiàng)批輸入接口，所以設(shè)計(jì)的MarcDc模塊應(yīng)在項(xiàng)批輸入者之前進(jìn)行處理。如果輸入的是MARC格式，則先經(jīng)過(guò)MarcDc格式轉(zhuǎn)換器進(jìn)行數(shù)據(jù)轉(zhuǎn)換，然后通過(guò)項(xiàng)批處理模塊進(jìn)行處理；如果輸入的就是DC格式，則直接應(yīng)用項(xiàng)批處理模塊進(jìn)行處理。3 OAI?PMH互操作平臺(tái)的實(shí)現(xiàn)3.1 基于OAI?PMH協(xié)議的命令動(dòng)詞分析基于OAI?PMH協(xié)議的數(shù)字圖書(shū)館中，為實(shí)現(xiàn)多個(gè)數(shù)據(jù)提供

4、者與服務(wù)提供者間之間的元數(shù)據(jù)交互，重新設(shè)計(jì)了基于OAI?PMH協(xié)議互操作平臺(tái)的命令動(dòng)詞，命令動(dòng)詞的主要功能如下：（1） GetRecord：此動(dòng)詞可從倉(cāng)儲(chǔ)中搜索元數(shù)據(jù)記錄。如果從倉(cāng)儲(chǔ)中和指定的項(xiàng)中無(wú)法得到由metadataPrefix指定的元數(shù)據(jù)格式，其可以返回狀態(tài)屬性的頭部信息值為“刪除”，此功能的實(shí)現(xiàn)依賴(lài)于倉(cāng)儲(chǔ)所跟蹤的刪除級(jí)別；（2） Identify：此動(dòng)詞用于檢索倉(cāng)儲(chǔ)的有關(guān)信息。利用此動(dòng)詞，倉(cāng)儲(chǔ)也可以返回余下的描述性信息，在返回的信息中，一部分對(duì)于基于OAI?PMH協(xié)議的互操作平臺(tái)是需要的；（3） ListIdentifiers：此動(dòng)詞不返回記錄本身，而僅返回頭部的記錄，通過(guò)基于集合成

5、員和時(shí)間戳的頭部可獲取選擇的參數(shù)?；趥}(cāng)儲(chǔ)對(duì)刪除的支持特性，如過(guò)請(qǐng)求中刪除了與指定參數(shù)匹配的記錄，則返回的記錄中將含有值為“刪除”的狀態(tài)屬性；（4） ListMetadataFormats：此動(dòng)詞可從倉(cāng)儲(chǔ)中檢索獲得所需的元數(shù)據(jù)格式，同時(shí)可以通過(guò)參數(shù)限制該請(qǐng)求對(duì)指定項(xiàng)元數(shù)據(jù)格式的獲取； （5） ListRecords：此動(dòng)詞用于從倉(cāng)儲(chǔ)中獲取記錄?；趥}(cāng)儲(chǔ)對(duì)刪除的支持特性，如果刪除了請(qǐng)求中與參數(shù)匹配的記錄，則返回的記錄中將含有值為“刪除”的狀態(tài)屬性，但返回有“刪除”狀態(tài)的記錄是不能被表達(dá)為元數(shù)據(jù)格式的；（6） ListSets：此動(dòng)詞用于返回倉(cāng)儲(chǔ)的集合結(jié)構(gòu)，對(duì)于選擇性獲取有益。3.2 數(shù)據(jù)提供者與

6、服務(wù)提供者間命令動(dòng)詞使用方式分析數(shù)據(jù)提供者與數(shù)據(jù)服務(wù)者之間的請(qǐng)求和應(yīng)答是通過(guò)上述六個(gè)核心動(dòng)詞來(lái)實(shí)現(xiàn)的。使用標(biāo)準(zhǔn)的Web服務(wù)器是一個(gè)典型請(qǐng)求的實(shí)現(xiàn)方式，通過(guò)配置該Web服務(wù)器，使其可以向能夠處理OAI?PMH請(qǐng)求的軟件分發(fā)OAI?PMH請(qǐng)求，具體交互過(guò)程如圖5所示。數(shù)據(jù)提供者與數(shù)據(jù)服務(wù)者之間的交互步驟具體如下：（1） 服務(wù)提供者首先找到其所需元數(shù)據(jù)的數(shù)據(jù)提供者，然后獲取其惟一標(biāo)識(shí)；（2） 服務(wù)提供者向數(shù)據(jù)提供者發(fā)出ListSets請(qǐng)求和ListMetaFormats請(qǐng)求。數(shù)據(jù)提供者根據(jù)服務(wù)提供者的請(qǐng)求返回其所能提供的滿(mǎn)足一定條件（如某種格式，某種主題，某個(gè)時(shí)間段）的元數(shù)據(jù)；服務(wù)提供者取得元數(shù)據(jù)后

7、向用戶(hù)提供服務(wù)。3.3 OAI?PMH命令動(dòng)詞的請(qǐng)求和應(yīng)答格式分析（1） 請(qǐng)求格式3.4 OAI六個(gè)動(dòng)詞的設(shè)計(jì)（1） 概要設(shè)計(jì)當(dāng)數(shù)據(jù)提供者接收到OAI請(qǐng)求后，需對(duì)該請(qǐng)求進(jìn)行解析。首先判斷該請(qǐng)求類(lèi)型是否合法，若不合法，則向服務(wù)提供者發(fā)送一個(gè)錯(cuò)誤的信息；若合法，則判斷該請(qǐng)求屬于六個(gè)有效請(qǐng)求的哪個(gè)類(lèi)型。由于參數(shù)metadataPrifix對(duì)于請(qǐng)求類(lèi)型 ListIdentifiers是強(qiáng)制的，所以如果數(shù)據(jù)提供者接收到的請(qǐng)求類(lèi)型是ListIdentifiers，那么解析器可以直接檢查這個(gè)請(qǐng)求的第二個(gè)參數(shù)。但如果請(qǐng)求中沒(méi)有相關(guān)參數(shù)，那么要確認(rèn)這個(gè)請(qǐng)求有效，則需確認(rèn)請(qǐng)求中必須包含resumptionToke

8、n （恢復(fù)標(biāo)志參數(shù)），并且數(shù)據(jù)提供者知道這個(gè)參數(shù)。假設(shè)數(shù)據(jù)提供者在unqualified DC模式下只能發(fā)送元數(shù)據(jù)集合，那么metadataPrifix參數(shù)惟一有效的值只能是oai_dc。在正常情況下，請(qǐng)求中可選取的參數(shù)是必須被解析的，但可以簡(jiǎn)單化，以一種非正式的形式進(jìn)行描述。然后，根據(jù)接收到的請(qǐng)求參數(shù)，數(shù)據(jù)提供者運(yùn)用SQL語(yǔ)句在倉(cāng)儲(chǔ)中進(jìn)行查詢(xún)，如果產(chǎn)生了記錄大于傳遞標(biāo)識(shí)符一次所能傳遞的最大值，那么數(shù)據(jù)提供者則會(huì)新生成一個(gè)resumptionToken標(biāo)志，并將查詢(xún)參數(shù)與指針信息存儲(chǔ)在一起，具體實(shí)現(xiàn)流程如圖6所示。（2） 詳細(xì)設(shè)計(jì) 利用Protocol數(shù)據(jù)庫(kù)包進(jìn)行六個(gè)動(dòng)詞的請(qǐng)求、響應(yīng)及信息顯示

9、，其中Date類(lèi)使用java.util.Date，Calendar或String方法實(shí)現(xiàn)年、月、日等各種格式間的相互轉(zhuǎn)換和輸出；Set類(lèi)可通過(guò)一個(gè)已命名的集合創(chuàng)建一個(gè)新的集合，并用XML形式描述。 處理客戶(hù)端源代碼Client，可以利用一個(gè)URL或一個(gè)集合的性質(zhì)創(chuàng)建HarvesterItinerary類(lèi)用于描述harvester類(lèi)的狀態(tài)，HarvesterItinerary類(lèi)中可以對(duì)Harvester類(lèi)的狀態(tài)進(jìn)行保存。OAIConnection類(lèi)可以描述一個(gè)到OAI的連接，用于接收單一的請(qǐng)求，該連接是基于用doReques或基于倉(cāng)儲(chǔ)的URL所返回的Response。 互操作平臺(tái)中的Server

10、包可以提供服務(wù)器端的OAI源代碼，并把源代碼轉(zhuǎn)換為文檔服務(wù)。OAI服務(wù)的目標(biāo)編程接口通過(guò)Target類(lèi)實(shí)現(xiàn)，互操作平臺(tái)通過(guò)TargetAdapter類(lèi)定義了一個(gè)不做任何事的執(zhí)行，GenericTarget類(lèi)可以實(shí)現(xiàn)由一個(gè)非常簡(jiǎn)單的機(jī)制以創(chuàng)建小型的OAI收集器，并在存儲(chǔ)器中尋找與服務(wù)，利用集合規(guī)程和給定的元數(shù)據(jù)前綴得到所有的記錄； JDBCServer類(lèi)是OAI servlet提供的一般服務(wù)器，OAIServerIfc類(lèi)為OAI提供了一個(gè)框架，元數(shù)據(jù)前綴用在DC元數(shù)據(jù)核心記錄，處理GetRecord 請(qǐng)求、Identify 請(qǐng)求、ListRecords請(qǐng)求、 ListSets請(qǐng)求、ListMet

11、adataFormats請(qǐng)求和ListIdentifiers請(qǐng)求。ResumableResultSet類(lèi)描述了一個(gè)結(jié)果，該結(jié)果通過(guò)與重用標(biāo)記相結(jié)合，可以被客戶(hù)作為一系列局部結(jié)果而重用。3.5 數(shù)據(jù)提供者的功能實(shí)現(xiàn)（1） 項(xiàng)的訂購(gòu)及提交用戶(hù)可以使用e?mail訂購(gòu)自己所需的項(xiàng)，這樣可以共享數(shù)字圖書(shū)館的資源。用戶(hù)首先提交項(xiàng)的要素描述元數(shù)據(jù)，然后上傳源文件，在通過(guò)系統(tǒng)的驗(yàn)證后可把文件提交到互操作平臺(tái)上供大家瀏覽及下載。（2） 工作流的實(shí)現(xiàn)三個(gè)小組負(fù)責(zé)人負(fù)責(zé)實(shí)現(xiàn)社團(tuán)的工作流，每個(gè)小組負(fù)責(zé)人完成不同的工作流步驟。工作流的次序如下：當(dāng)社團(tuán)收到一個(gè)遞呈時(shí)，如果社團(tuán)中有小組負(fù)責(zé)人，那么他將會(huì)選擇接受或拒絕，如

12、果社團(tuán)中沒(méi)有小組負(fù)責(zé)人，這一步工作流將會(huì)被直接省略。第二步和第三步的遞呈也是按照此步驟處理。當(dāng)調(diào)用工作流的第一步時(shí)，完成工作流步驟的任務(wù)把相關(guān)的遞呈放到“任務(wù)箱”中，如果組中的一個(gè)成員接受任務(wù)箱中的任務(wù)，則任務(wù)被從任務(wù)箱中移出。如果一個(gè)遞呈被拒絕，則系統(tǒng)會(huì)通過(guò)e?mail把原因發(fā)送給遞呈提交者，提交者可以修改后重新提交；如果一個(gè)遞呈被“接受”，將轉(zhuǎn)到工作流的下一步。（3） 搜索和瀏覽功能實(shí)現(xiàn)終端用戶(hù)可以使用多種方法發(fā)現(xiàn)內(nèi)容，具體如下：使用關(guān)鍵字進(jìn)行搜索；使用外部的ID號(hào)進(jìn)行搜索；使用標(biāo)題、數(shù)據(jù)進(jìn)行瀏覽。在數(shù)字圖書(shū)館系統(tǒng)中，發(fā)現(xiàn)內(nèi)容最基本的方法是搜索。基于OAI?PMH協(xié)議的互操作平臺(tái)的搜索和索

13、引模塊是一個(gè)簡(jiǎn)單的API接口，它能夠在全部的社區(qū)、社團(tuán)中完成新內(nèi)容的索引、再生索引，系統(tǒng)的具體搜索界面如圖7所示。 4 結(jié) 論本文設(shè)計(jì)了MARC格式到DC格式的格式轉(zhuǎn)換器及基于OAI?PMH協(xié)議互操作平臺(tái)的命令動(dòng)詞，解決了數(shù)字圖書(shū)館中數(shù)據(jù)提供者與服務(wù)提供者間的元數(shù)據(jù)交互問(wèn)題。給出了不同格式間的對(duì)照關(guān)系及格式轉(zhuǎn)換器的實(shí)現(xiàn)方法，同時(shí)完成了命令動(dòng)詞的實(shí)現(xiàn)代碼，對(duì)推動(dòng)OAI?PMH協(xié)議在數(shù)字圖書(shū)館領(lǐng)域的應(yīng)用具有重要意義。With the rapid development of network, computer and communication technology, digital librar

14、y has become the important sources of information for people to obtain information, but for many distributed digital resources on the network 1? 2, it is hoped that through to retrieve the required information. But since early in the establishment of digital library, without unified standard, its re

15、source metadata format is not the same, and thus to develop a set of can describe the unity of the different digital library resources metadata format is very difficult; Early at the same time, set up the digital library of data between the provider and the service provider is a point-to-point proto

16、col service, outside of the agreement between data providers and service providers to metadata interaction 3? 4.According to the above problem, design is widely used in the digital library metadata format format of MARC and DC converter, and redesigned based on OAI? PMH protocol interoperability pla

17、tform command verbs, between the digital library resources sharing is of great significance.1 OAI? PMH protocol overviewData providers and service providers, registered server constitutes the OAI in digital library field ? 5 7? PMH the framework of the agreement. Data providers are stored in the fra

18、mework of a large number of metadata repository, the service provider can extract meta data from multiple data provider and provide searching, browsing and other value-added services, the service provider is sent to the data provider to UDP request, which is based on OAI? PMH command verbs, a data p

19、rovider to the service provider's request is through HTTP protocol in the form of XML response, registered in the server can be the registration data providers and service providers. The specific framework is shown in figure 1.2, MARC format to DC converter design and implementationCurrently use

20、d in the digital library metadata format to MARC, and based on OAI? PMH protocol data providers can only support DC format metadata 8. So in order to realize based on OAI? PMH protocol metadata interoperability platform, you need to put the metadata of MARC format is converted to DC metadata format.

21、2.1 comparison relationship of MARC and DC format domainDC format contains element describes a total of 15, according to the scope and categories describe content can be divided into three groups, as shown in table 1.This system provides a number of input interface, so the design of MarcDc module sh

22、ould be performed before the item number of input processing. If the input is MARC format, after first MarcDc format converter to convert the data, then by a batch processing module; If the input is a DC format, the direct application of a batch processing module for processing.3 OAI? The realizatio

23、n of PMH interoperable platform3.1 based on OAI? PMH protocol analysis of the command verbsBased on the OAI? PMH agreement in the digital library, in order to realize multiple data interaction between metadata between providers and service providers, redesigned based on OAI? PMH protocol interoperab

24、ility platform command verbs, the main functions of the command verbs are as follows:GetRecord (1) : this verb can search metadata record from the warehouse. If the specified item from the warehouse and can't get appointed by metadataPrefix metadata format, the head of the state can return prope

25、rty information value for the "delete", the realization of the function of this depends on the storage by tracking the deletion of level;(2) Identify: this verb is used to retrieve information about the warehouse. Using the verb, storage can also return to the rest of the descriptive infor

26、mation, in the information returned part based on OAI? PMH protocol interoperability platform is needed;(3) ListIdentifiers: this verb does not return the record itself, and records of return only the head, by head based on the set members and timestamp available choice of parameters. Based on wareh

27、ouse to remove the support features, such as a request to delete a record with the specified parameters matching, it returns the record will contain values for the "delete" state property;(4) ListMetadataFormats: this verb can be retrieved from the storage to obtain the required metadata f

28、ormat, at the same time can through parameter limits the request to specify item metadata format for; (5) ListRecords: extracted from the verbs used in the warehouse record. Support feature based on warehouse to delete, if you remove the request and parameter matching of records, it returns the reco

29、rd will contain values for the state of the "delete" attribute, but returned to a state of "delete" record cannot be expressed as metadata format;(6) ListSets: the collection structure of verbs used to return to the warehouse, to get good selectivity.3.2 between data providers an

30、d service providers command verb usage analysisBetween data providers and data service request and response is done by the above six core verb. Use the standard Web server implementation approach, is a typical request by configuring the Web server, allows it to be able to handle OAI? PMH request sof

31、tware distribution of OAI? PMH requests, the specific interaction process as shown in figure 5.Interaction between data providers and data server steps specific as follows:(1) the service provider first find the metadata for the data provider, and then get its unique identity;(2) the service provide

32、r to the data provider ListSets requests and ListMetaFormats request. Data provider according to the service provider's request to return to its can provide satisfy certain conditions (such as some format, a theme, a certain period of time) metadata; After the service provider to obtain metadata

33、 to provide service to the users.3.3 OAI? PMH command request and response format analysis of verbs(1) request format3.4 the design of the OAI six verbs(1) the profile designWhen the data provider receives the OAI request, need to parse the request. First determines whether the request type is legal

34、, if not legal, then send a wrong message to the service provider; If legally, it concludes that the request which belong to six effective request type. As the parameter metadataPrifix for request types ListIdentifiers is mandatory, so if the data provider receives the request type is ListIdentifier

35、s, so the parser can directly check the request of the second parameter. But if there is no related parameters in the request, so want to confirm that the request is valid, will be expected to confirm the request must contain resumptionToken (recovery mark parameters), and the data provider know thi

36、s parameter.The assumption that data provider in unqualified DC mode can only send the metadata collection, so the only effective value can only be oai_dc metadataPrifix parameters. Under normal circumstances, can select the parameters in the request must be resolved, but can be simple, in the form

37、of an informal description. Then, according to the received request parameters, the data provider query using SQL statements in the warehouse, if the record is more than a passing identifier can pass the maximum, then the data provider will generate a new resumptionToken logo, and the query paramete

38、r with pointer information is stored in the concrete implementation process is shown in figure 6.(2) the detailed design(1) using Protocol data packet six verbs, according to the request, response, and the information of the Date class using Java. Util. The Date, year, month, day Calendar or String

39、method between the various formats, such as mutual conversion and output; The Set class by a named Set to create a new Set of formal description using XML.(2) to deal with the Client source code, the Client, can use a URL or a collection of properties to create HarvesterItinerary class used to descr

40、ibe the state of the harvester class, HarvesterItinerary class the harvester state of a class can be saved. OAIConnection class can describe a connection to the OAI, to accept a single request, the connection is based on using doReques or URL returned by the Response based on warehouse.(3) interoper

41、ability platform of the OAI source code package can provide the Server Server, and convert the source code to document service. OAI service Target programming interface on a Target class implements the interoperability platform through TargetAdapter class defines a don't do anything, GenericTarg

42、et class can implement by a very simple mechanism to create a small OAI collector, and are looking for and service, in the memory collection procedures and the given metadata prefixes are used to get the all records; JDBCServer class is to provide general server of OAI servlet OAIServerIfc such as O

43、AI provides a framework, core metadata prefix used in DC metadata records, processing GetRecord request, the Identify, ListRecords request, ListSets, ListMetadataFormats request with ListIdentifiers request. ResumableResultSet class describes a result, the result by combined with reuse marker, can b

44、e reused. As a result of the series of local customer3.5 the realization of the function of the data providerItem (1) of the order and submitThe user can use the e? Mail you need to order the items that we can share the resources of the digital library. User submitted item first describes the elemen

45、ts of metadata, then upload the source file, through the system after the validation of documents submitted to interoperable platform for everyone to browse and download.(2) the realization of the workflowThree team leader is responsible for implementing corporate workflow, each team leader to compl

46、ete different workflow steps. The order of the workflow is as follows: when the club received a presented, if a team leader in the community, so he will choose to accept or reject, if there is no team leader in the community, will be directly omit this step workflow. The second and third steps presented is also in accordance with this step.When calli