Welcome to LangSearch’s documentation!

• Introduction
• Quickstart
  • Introduction
  • Crawling selected pages of the LangChain documentation
  • Removing boilerplate, extracting text, splitting text and indexing
  • Running the crawler
  • Create a QA app
  • Keep your QA app up-to-date
• Installation
• Gathering Data
  • Introduction
  • Creating a Scrapy project
  • Crawling websites
    • WebSpider settings
    • Running the crawler
    • Checking crawl behavior using the DryRunPipeline
    • What does WebSpider send to the item pipeline?
    • Restricting the crawl to a particular domain
  • Gathering data from local filesystem
• Indexing Data
  • Introduction
  • Built-in pipelines
• Pipelines
  • GenericHTMLPipeline
    • Service requirements
  • GenericPlainTextPipeline
    • Service requirements
  • GenericAudioPipeline
    • Service requirements
  • GenericImagePipeline
    • Service requirements
  • GenericOtherPipeline
    • Service requirements
• Examples
  • Audio Data
  • Image Data
  • PDF, DOCX & co
• Customizing and extending