> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pal.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Create Documents by Scraping URLs

> This endpoint lets you pass in one or more URLs that we will then fetch. Optionally you can specify a contentSelector that is a DOM Selector we ought to use to find the relevant text from the page.



## OpenAPI

````yaml POST /documents/from-url
openapi: 3.1.0
info:
  title: HeyPal API
  version: '1.0'
  description: >-
    The HeyPal API makes it easy to add A.I. powered semantic-search & other
    A.I. functionality to your product & content.
servers:
  - url: https://api.pal.ai/v1
security:
  - API Key: []
paths:
  /documents/from-url:
    post:
      summary: Create Documents by Scraping URLs
      description: >-
        This endpoint lets you pass in one or more URLs that we will then fetch.
        Optionally you can specify a contentSelector that is a DOM Selector we
        ought to use to find the relevant text from the page.
      operationId: post-documents-from-url
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                applicationId:
                  type: string
                  x-stoplight:
                    id: 36dvhvixceu3r
                urls:
                  type: array
                  x-stoplight:
                    id: 8v8xxy4cmi58k
                  items:
                    x-stoplight:
                      id: a9nxzt6nddug4
                    type: string
                    example: '"techmeme.com/article1"'
                contentSelector:
                  type: string
                  x-stoplight:
                    id: 0hohgbqvpvecs
                metadata:
                  type: string
                  x-stoplight:
                    id: ww4jk3ekn6xru
              required:
                - applicationId
                - urls
        description: 'Pass '
      responses:
        '200':
          description: OK
components: {}

````