Apache Solr Beyond The Box

ApacheSolrBeyondTheBoxChrisHostetter2008-11-05~hossman/apachecon2008us/!●What,How,Where,When,Why?●SolrInternalsInANutshell●RealWorldExamples●Testing●Questions3What,How,Where,Who,When,Why?4WhatIsSolr(ToUsers)●InformationRetrievalApplication●Index/QueryViaHTTP●ComprehensiveHTMLAdministrationInterfaces●Scalability-EfficientReplicationToOtherSolrSearchServers●HighlyConfigurableCaching●FlexibleAndAdaptableWithXMLConfigurationCustomizableRequestHandlersAndResponseWritersDataSchemaWithDynamicFieldsAndUniqueKeysAnalyzersCreatedAtRuntimeFromTokenizersAndTokenFiltersWhatIsSolr(ToDevelopers)●InformationRetrievalApplication●Java5WebApp(WAR)WithAWebServices-ishAPI●ExtensiblePluginArchitecture●MVC-ishFrameworkAroundTheJavaLuceneSearchLibrary●AllowsCustomBusinessLogicandTextAnalysisRulesToLiveCloseToTheData●AbstractsAwayTheTrickyStuff:IndexConsistencyDataReplicationCacheManagementHowItStartedWhen/WhyToWriteAPlugin“Xcanbedonemoreefficientlyclosertothedata.”OR“ToforceXforallclients.”8SolrInternalsInANutshell950,000'ViewHTTPSolrDispatchFilterJavaEmbeddedSolrServerSolrCoreSolrCoreSolrCoreSolrRequestHandlerCoreContainerSolrQuery(Request/Response)QueryResponseWriterMVC-ish●SolrRequestHandler...AControllerhandleRequest(SolrQueryRequest,SolrQueryResponse )●SolrQueryRequest...AnEvent(++)InputParametersListofContentStreamsMaintainsSolrCore&SolrIndexSearcherReferences●SolrQueryResponse...ModelTreeofSimpleObjectsandDocLists●ResponseWriter...Viewwrite(Writer,SolrQueryRequest,SolrQueryResponse)11public class HelloWorld extends RequestHandlerBase {  public void handleRequestBody(SolrQueryRequest req,                                SolrQueryResponse rsp) {    String name = req.getParams().get(name);    Integer age = req.getParams().getInt(age);    rsp.add(greeting, Hello  + name);    rsp.add(yourage, age);  }  public String getVersion() { return $Revision:$; }  public String getSource() { return $Id:$; }  public String getSourceId() { return $URL:$; }  public String getDescription() { return Says Hello; }}HelloWorld12=Hoss&age=32&wt=xml    response      lst name=responseHeader        int name=status0/int        int name=QTime1/int      /lst      str name=greetingHello Hoss/str      int name=yourage32/int    /response=Hoss&age=32&wt=json    { responseHeader:{ status:0, Qtime:1},      greeting:Hello Hoss,      yourage:32    }HelloWorldOutputTypesOfPlugins●SolrRequestHandlerSolrRequestHandlerSearchComponentSearchComponentQparserPluginQparserPluginValueSourceParserValueSourceParser●SolrHighlighterSolrHighlighterSolrFragmenterSolrFragmenterSolrFormatterSolrFormatter●UpdateRequestProcessorFactoryUpdateRequestProcessorFactory●QueryResponseWriterQueryResponseWriterItalics:OnlyOnePerSolrCoreCCololoror:LikelihoodOfNeedingToWriteYourOwn●Similarity(Factory)Similarity(Factory)●AnalyzerAnalyzerTokenizerFactoryTokenizerFactoryTokenFilterFactoryTokenFilterFactory●FieldTypeFieldType●SolrCacheSolrCacheCacheRegeneratorCacheRegenerator●SolrEventListenerSolrEventListener●UpdateHandlerUpdateHandler14RealWorldExamples15TibetanAndHimalayanDigitalLibraryTools16   public class TshegBarTokenizerFactory                 extends BaseTokenizerFactory {     public TokenStream create(Reader input) {       return new TshegBarTokenizer(input);     }   }   public class EdgeTshegTrimmerFactory                 extends BaseTokenFilterFactory {       public TokenStream create(TokenStream input) {           return new EdgeTshegTrimmer(input);       }   }TshegAnalysisFactories17DFLLDFLL:FacetedBrowsingDFLLCategoryMetadata●CategoryIDandLabel:3126 == “Tablet PCs”●CategoryQuery:tablet_form:[* TO *]●OrderedListofFacetsFacetIDandLabel:500016 == “OS Provided”FacetDisplayInfo:Countvs.Alphabetical,etc...OrderedListofConstraints●ConstraintIDandLabel:111536 == “Apple OS X”●ConstraintQuery:os:(“OSX10.1” “OSX10.2” ...)20Document catMetaDoc = searcher.getFirstMatch(catDocId)Metadata m = parseAndCacheMetadata(catMetaDoc, searcher)m = m.clone()DocListAndSet results =              searcher.getDocListAndSet(m.catQuery, ...)response.add(“products”, results.docList)foreach (Facet f : m) {  foreach (Constraint c : f) {    c.setCount(searcher.numDocs(c.query,                                results.docSet))  }}response.add(“metadata”, m.asSimpleObjects())DfllHandlerPsuedo-CodeConceptualPictureDocListgetDocListAndSet(Query,Query[],Sort,offset,n)os:(“OSX10.1”“OSX10.2”...)memory:[1GBTO*]tablet_form:[*TO*]priceascproc_manu:Intelproc_manu:AMDSectionoforderedresultsDocSetUnorderedsetofallresultsprice:[0TO500]price:[500TO1000]manu:Dellmanu:HPmanu:LenovonumDocs()=594=382=247=689=104=92=75QueryResponse22result name=products numFound=394 start=0.../resultslst name=metadata ... lst name=500016   int name=rankDir0/intint name=datatype1/int   int name=rating88/intstr name=nameOS provided/str   l