Querying

DDO: Querying

Querying in DDO uses comparison and logical operators to find documents that match specific criteria.

Querying are particularly useful for:

• Filtering lists of objects/dictionaries
• Searching through complex nested data structures
• Building dynamic query interfaces
• Creating data pipelines with filtering stages
• Simplifying data retrieval logic in applications

Query Syntax

Filters support two syntax styles for specifying field paths: flat and nested.

Flat Syntax (With Dot Notation)

{
    "field:$operator": value,
    "nested.path.to.field:$operator": value
}

Where:

• field is the property name to filter on (can use dot notation for nested fields)
• $operator is one of the supported filter operators
• value is the comparison value to filter against

For simple equality checks, you can use the shorthand:

{
    "field": value,  # Equivalent to "field:$eq": value
    "nested.path.to.field": value
}

Nested Syntax (With Nested Dictionaries)

You can also use nested dictionaries to represent paths:

{
    "parent": {
        "child": {
            "grandchild:$operator": value
        }
    }
}

This is equivalent to the flat syntax:

{
    "parent.child.grandchild:$operator": value
}

Important: This syntax supports nesting for field paths only. The operator is always attached to the field name with a colon, not as a separate nested object.

Example Testing Nested Structure

Let's look at an example that demonstrates filtering with a deeply nested structure:

# Sample data
data = [
    {
        "name": {
            "deep": {
                "datalength": 3,
                "values": ["a", "b", "c"]
            }
        },
        "status": "active"
    },
    {
        "name": {
            "deep": {
                "datalength": 7,
                "values": ["a", "b", "c", "d", "e", "f", "g"]
            }
        },
        "status": "pending"
    },
    {
        "name": {
            "deep": {
                "datalength": 2,
                "values": ["x", "y"]
            }
        },
        "status": "active"
    }
]

# Using flat syntax
flat_filter = {
    "name.deep.datalength:$gt": 5
}

# Using nested syntax
nested_filter = {
    "name": {
        "deep": {
            "datalength:$gt": 5
        }
    }
}

# Both filters will return the same item (the second one with datalength 7)

This example shows how both syntaxes target the same nested data effectively.

---

Basic Comparison Operators

$eq - Equal To

Find documents where a field equals a specific value.

# Find users named "John"
{
    "name:$eq": "John"
}

# Alternative shorthand (implicit $eq)
{
    "name": "John"
}

# Find users with exactly 5 login attempts
{
    "loginAttempts:$eq": 5
}

$ne - Not Equal To

Find documents where a field does not equal a specific value.

# Find users who are not banned
{
    "status:$ne": "banned"
}

# Find products that aren't out of stock
{
    "inventory.quantity:$ne": 0
}

$gt - Greater Than

Find documents where a field is greater than a value.

# Find users older than 25
{
    "age:$gt": 25
}

# Find orders with value greater than $100
{
    "total:$gt": 100.00
}

$gte - Greater Than or Equal

Find documents where a field is greater than or equal to a value.

# Find users 18 or older
{
    "age:$gte": 18
}

# Find high-priority items (priority 7 or above)
{
    "priority:$gte": 7
}

$lt - Less Than

Find documents where a field is less than a value.

# Find junior employees (less than 2 years experience)
{
    "experience:$lt": 2
}

# Find small orders (less than $50)
{
    "total:$lt": 50.00
}

$lte - Less Than or Equal

Find documents where a field is less than or equal to a value.

# Find users 65 or younger
{
    "age:$lte": 65
}

# Find budget items ($25 or less)
{
    "price:$lte": 25.00
}

Array and Collection Operators

$in - Value In Array

Find documents where a field's value is in a specified array.

# Find users with specific roles
{
    "role:$in": ["admin", "moderator", "editor"]
}

# Find products in certain categories
{
    "category:$in": ["electronics", "computers", "mobile"]
}

$nin - Value Not In Array

Find documents where a field's value is not in a specified array.

# Find users who are not in restricted roles
{
    "role:$nin": ["banned", "suspended", "deleted"]
}

# Find products not in discontinued categories
{
    "category:$nin": ["discontinued", "obsolete"]
}

$all - All Values Match

Find documents where an array field contains all specified values.

# Find users with all required permissions
{
    "permissions:$all": ["read", "write", "delete"]
}

# Find products with all required tags
{
    "tags:$all": ["featured", "bestseller"]
}

$size - Array Size

Find documents where an array has a specific length.

# Find users with exactly 3 skills
{
    "skills:$size": 3
}

# Find orders with exactly 2 items
{
    "items:$size": 2
}

String Operators

$contains - Contains Substring

Find documents where a string field contains a substring.

# Find users with "gmail" in their email
{
    "email:$contains": "gmail"
}

# Find products with "wireless" in description
{
    "description:$contains": "wireless"
}

$startswith - Starts With

Find documents where a string field starts with a prefix.

# Find users with names starting with "John"
{
    "name:$startswith": "John"
}

# Find phone numbers starting with country code
{
    "phone:$startswith": "+1"
}

$endswith - Ends With

Find documents where a string field ends with a suffix.

# Find files with .pdf extension
{
    "filename:$endswith": ".pdf"
}

# Find email addresses from specific domain
{
    "email:$endswith": "@company.com"
}

Pattern Matching

$match - Pattern Matching

Find documents using wildcard patterns or regex.

# Wildcard patterns (* for multiple chars, ? for single char)
{
    "filename:$match": "report_*.pdf"
}

# Find names with specific pattern
{
    "code:$match": "PRD-????-2023"
}

# Regex patterns (enclosed in forward slashes)
{
    "email:$match": "/^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$/"
}

$regex - Regular Expression

Find documents using regular expressions.

# Find phone numbers in specific format
{
    "phone:$regex": "^\\+1-\\d{3}-\\d{3}-\\d{4}$"
}

# Find postal codes
{
    "zipcode:$regex": "^\\d{5}(-\\d{4})?$"
}

Existence and Type Checking

$exists - Field Exists

Find documents where a field exists or doesn't exist.

# Find users with a profile picture
{
    "profilePicture:$exists": True
}

# Find users without a middle name
{
    "middleName:$exists": False
}

$type - Type Checking

Find documents where a field is of a specific type.

# Find documents where age is a number
{
    "age:$type": "number"
}

# Find documents where tags is an array
{
    "tags:$type": "array"
}

# Supported types: string, number, boolean, array, object, null

Complex Filtering with JMESPath

DDO supports JMESPath expressions for complex filtering scenarios.

# Find admin users
{
    "users[?role == 'admin'].name": "*"
}

# Find active orders over $100
{
    "orders[?status == 'active' && total > 100].id": "*"
}

# Find users in specific age range
{
    "users[?age >= 25 && age <= 65].email": "*"
}

Combining Filters

You can combine multiple filter conditions in a single QLO:

# Find active adult users with gmail accounts
{
    "status": "active",
    "age:$gte": 18,
    "email:$contains": "gmail"
}

# Find premium products in stock within price range
{
    "category": "premium",
    "stock:$gt": 0,
    "price:$gte": 100,
    "price:$lte": 500
}

---