Scaling Big Data with Hadoop and Solr -- Table of Contents -- Scaling Big Data with Hadoop and Solr -- Credits -- About the Author -- About the Reviewer -- www.PacktPub.com -- Support files, eBooks, discount offers and more -- Why Subscribe? -- Free Access for Packt account holders -- Preface -- What this book covers -- What you need for this book -- Who this book is for -- Conventions -- Reader feedback -- Customer support -- Downloading the example code -- Errata -- Piracy -- Questions -- 1. Processing Big Data Using Hadoop and MapReduce -- Understanding Apache Hadoop and its ecosystem -- The ecosystem of Apache Hadoop -- Apache HBase -- Apache Pig -- Apache Hive -- Apache ZooKeeper -- Apache Mahout -- Apache HCatalog -- Apache Ambari -- Apache Avro -- Apache Sqoop -- Apache Flume -- Storing large data in HDFS -- HDFS architecture -- NameNode -- DataNode -- Secondary NameNode -- Organizing data -- Accessing HDFS -- Creating MapReduce to analyze Hadoop data -- MapReduce architecture -- JobTracker -- TaskTracker -- Installing and running Hadoop -- Prerequisites -- Setting up SSH without passphrases -- Installing Hadoop on machines -- Hadoop configuration -- Running a program on Hadoop -- Managing a Hadoop cluster -- Summary -- 2. Understanding Solr -- Installing Solr -- Apache Solr architecture -- Storage -- Solr engine -- The query parser -- Interaction -- Client APIs and SolrJ client -- Other interfaces -- Configuring Apache Solr search -- Defining a Schema for your instance -- Configuring a Solr instance -- Configuration files -- Request handlers and search components -- Facet -- MoreLikeThis -- Highlight -- SpellCheck -- Metadata management -- Loading your data for search -- ExtractingRequestHandler/Solr Cell -- SolrJ -- Summary -- 3. Making Big Data Work for Hadoop and Solr -- The problem -- Understanding data-processing workflows.

The standalone machine -- Distributed setup -- The replicated mode -- The sharded mode -- Using Solr 1045 patch - map-side indexing -- Benefits and drawbacks -- Benefits -- Drawbacks -- Using Solr 1301 patch - reduce-side indexing -- Benefits and drawbacks -- Benefits -- Drawbacks -- Using SolrCloud for distributed search -- SolrCloud architecture -- Configuring SolrCloud -- Using multicore Solr search on SolrCloud -- Benefits and drawbacks -- Benefits -- Drawbacks -- Using Katta for Big Data search (Solr-1395 patch) -- Katta architecture -- Configuring Katta cluster -- Creating Katta indexes -- Benefits and drawbacks -- Benefits -- Drawbacks -- Summary -- 4. Using Big Data to Build Your Large Indexing -- Understanding the concept of NOSQL -- The CAP theorem -- What is a NOSQL database? -- The key-value store or column store -- The document-oriented store -- The graph database -- Why NOSQL databases for Big Data? -- How Solr can be used for Big Data storage? -- Understanding the concepts of distributed search -- Distributed search architecture -- Distributed search scenarios -- Lily - running Solr and Hadoop together -- The architecture -- Write-ahead Logging -- The message queue -- Querying using Lily -- Updating records using Lily -- Installing and running Lily -- Deep dive - shards and indexing data of Apache Solr -- The sharding algorithm -- Adding a document to the distributed shard -- Configuring SolrCloud to work with large indexes -- Setting up the ZooKeeper ensemble -- Setting up the Apache Solr instance -- Creating shards, collections, and replicas in SolrCloud -- Summary -- 5. Improving Performance of Search while Scaling with Big Data -- Understanding the limits -- Optimizing the search schema -- Specifying the default search field -- Configuring search schema fields -- Stop words -- Stemming -- Index optimization.

Limiting the indexing buffer size -- When to commit changes? -- Optimizing the index merge -- Optimize an option for index merging -- Optimizing the container -- Optimizing concurrent clients -- Optimizing the Java virtual memory -- Optimization the search runtime -- Optimizing through search queries -- Filter queries -- Optimizing the Solr cache -- The filter cache -- The query result cache -- The document cache -- The field value cache -- Lazy field loading -- Optimizing search on Hadoop -- Monitoring the Solr instance -- Using SolrMeter -- Summary -- A. Use Cases for Big Data Search -- E-commerce websites -- Log management for banking -- The problem -- How can it be tackled? -- High-level design -- B. Creating Enterprise Search Using Apache Solr -- schema.xml -- solrconfig.xml -- spellings.txt -- synonyms.txt -- protwords.txt -- stopwords.txt -- C. Sample MapReduce Programs to Build the Solr Indexes -- The Solr-1045 patch - map program -- The Solr-1301 patch - reduce-side indexing -- Katta -- Index.