OpenSearch

Connect your OpenSearch instance with NOFire AI

Connecting your OpenSearch instance to NOFire AI enables powerful querying and analysis of application and infrastructure logs for causal root cause analysis and predictive reliability.

NOFire AI supports both AWS OpenSearch Service and self-hosted OpenSearch instances using JWT bearer token authentication for secure, standardized access.

Prerequisites

Before connecting OpenSearch to NOFire AI, ensure you have:

• An OpenSearch cluster (AWS OpenSearch Service version 2.11+ or self-hosted)
• Network access to your OpenSearch endpoint from NOFire AI
• Admin access to configure authentication
• For AWS OpenSearch: Fine-grained access control enabled

Step 1: Configure OpenSearch Authentication

Choose the authentication method based on your OpenSearch deployment:

Option A: AWS OpenSearch Service (JWT Authentication - Recommended)

AWS OpenSearch Service supports JWT authentication for secure, token-based access. This requires OpenSearch version 2.11 or later with fine-grained access control enabled.

1. Generate an RSA Key Pair

Generate an RSA public/private key pair using OpenSSL:

# Generate private key
openssl genrsa -out privatekey.pem 2048

# Extract public key
openssl rsa -in privatekey.pem -pubout -out publickey.pem

The publickey.pem file contains your public key, and privatekey.pem contains the private key for signing JWTs.

You can also use ECDSA encryption. Both RSA and ECDSA asymmetric algorithms are supported. Learn more in the AWS OpenSearch JWT documentation.

2. Update Domain Access Policy

Before enabling JWT authentication, update your domain access policy to allow JWT-authenticated requests:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:ESHttp*",
      "Resource": "arn:aws:es:us-east-1:111122223333:domain/domain-name/*"
    }
  ]
}

3. Enable JWT Authentication

In the AWS Console or using the AWS CLI:

Via AWS Console:

1. Navigate to Amazon OpenSearch Service > Select your domain
2. Under Domain configuration, find JWT authentication and authorization for OpenSearch
3. Select Enable JWT authentication and authorization
4. Upload your publickey.pem file or paste its content
5. (Optional) Configure Subject key (default: sub) and Roles key (default: roles)
6. Save the changes

Via AWS CLI:

aws opensearch update-domain-config \
  --domain-name your-domain-name \
  --advanced-security-options '{
    "JWTOptions": {
      "Enabled": true,
      "PublicKey": "<paste-your-public-key-content>",
      "SubjectKey": "sub",
      "RolesKey": "roles"
    }
  }'

4. Create a JWT Token

Generate a JWT token signed with your private key. The token must include:

• Subject (sub): A valid OpenSearch user
• Roles (roles): Array of OpenSearch roles (e.g., ["readall"])

You can use tools like jwt.io, libraries (PyJWT, jsonwebtoken), or your identity provider to create the token.

Example JWT payload:

{
  "sub": "nofireai-reader",
  "roles": ["readall"],
  "iat": 1609459200,
  "exp": 1640995200
}

Sign this payload with your privatekey.pem using RS256 or ES256 algorithm.

Ensure the user (sub) and roles exist in your OpenSearch domain with appropriate permissions. Create them via OpenSearch Dashboards Security panel if needed.

5. Note Your Connection Details

You’ll need:

• OpenSearch Endpoint: https://search-your-domain.region.es.amazonaws.com
• JWT Token: The signed token from step 4

Option B: Self-Hosted OpenSearch (Bearer Token Authentication)

For self-hosted OpenSearch instances, use JWT bearer token authentication.

1. Configure JWT Authentication

Edit your opensearch.yml to enable JWT authentication:

plugins.security.auth.type: "jwt"
plugins.security.jwt.key: "<your-jwt-signing-key>"
plugins.security.jwt.header: "Authorization"

2. Create Service Account or User

Access OpenSearch Dashboards and create an internal user for the token:

1. Navigate to OpenSearch Dashboards (typically https://your-opensearch-host:5601)
2. Go to Security > Internal Users
3. Click Create internal user
4. Set username: nofireai-logs-reader
5. Set as service account with "service": "true" attribute
6. Assign appropriate roles (e.g., readall)

3. Generate Bearer Token

Create a JWT token with appropriate claims or use your identity provider’s token. The token should include:

• User identity claims: Subject matching the OpenSearch user
• Role/permission information: Roles array with read permissions

For detailed JWT configuration on self-hosted OpenSearch, see the OpenSearch Authentication Tokens documentation.

4. Note Your Connection Details

You’ll need:

• OpenSearch URL: https://opensearch.mycompany.com:9200
• Bearer Token: Your JWT token

Step 2: Add OpenSearch Connection to NOFire AI

Navigate to the Connections Tab

1. Select OpenSearch: From the list of available connections

   

   Then, click Next at the top right corner.

2. Enter Connection Details:

   • Connection name: Enter a descriptive name (e.g., “production-opensearch”)
   • URL: Your OpenSearch endpoint
     • AWS Example: https://search-my-domain-abc123.us-east-1.es.amazonaws.com
     • Self-hosted Example: https://opensearch.mycompany.com:9200
   • Token: The JWT token you created in Step 1
     • For AWS OpenSearch: The JWT token from Option A
     • For Self-Hosted: The bearer token from Option B

   

3. Test and Save: Click Test Connection to verify the credentials, then Save to create the connection.

   Once the connection to your log source is established, NOFire AI will query and analyze logs on-demand as part of its causal root cause analysis. Logs are queried only when needed to enrich investigations.

Additional Resources

• AWS OpenSearch JWT Authentication Documentation
• OpenSearch Security Authentication Tokens