Skip to content

AsaiYusuke/jsonpath

Repository files navigation

AsaiYusuke/JSONPath

Test Go Report Card Coverage Status Go Reference Awesome Go License: MIT

AsaiYusuke/JSONPath

This Go library allows you to extract parts of a JSON object using the JSONPath query syntax.

The core JSONPath syntax supported by this library is based on:

Note

For details on syntax compatibility with other libraries, see πŸ“ my comparison results.

Table of Contents

Getting started

Install

go get github.com/AsaiYusuke/jsonpath/v2

Simple example

package main

import (
  "encoding/json"
  "fmt"

  "github.com/AsaiYusuke/jsonpath/v2"
)

func main() {
  jsonPath, srcJSON := `$.key`, `{"key":"value"}`
  var src any
  json.Unmarshal([]byte(srcJSON), &src)
  output, _ := jsonpath.Retrieve(jsonPath, src)
  outputJSON, _ := json.Marshal(output)
  fmt.Println(string(outputJSON))
  // Output:
  // ["value"]
}

Basic design

Streamlined Development

  • The JSONPath syntax parser is implemented using PEG, which helps keep the source code simple and maintainable.
  • Robust unit tests are provided to prevent bugs and ensure consistent results.

User-Friendly Interface

  • The library provides comprehensive error types, making it easy for users to handle errors appropriately.

High Compatibility

How to use

* Retrieve one-time or repeatedly

The Retrieve function extracts values from a JSON object using a JSONPath expression:

output, err := jsonpath.Retrieve(jsonPath, src)

πŸ“ Example

The Parse function returns a parser function that checks the JSONPath syntax in advance. You can use this parser function to repeatedly extract values with the same JSONPath:

parsed, err := jsonpath.Parse(jsonPath)
output1, err1 := parsed(src1)
output2, err2 := parsed(src2)

πŸ“ Example

parser function

  • The parser function accepts an optional second argument: a pointer to a []any buffer.
  • When omitted or nil, a new slice is allocated on each call and returned.
  • When provided (non-nil), the buffer will be reset to length 0 and filled directly, enabling reuse without extra allocations.

Example of buffer reuse:

parsed, _ := jsonpath.Parse("$[*]")
buf := make([]any, 0, 4)
args := []*[]any{&buf}
out1, _ := parsed([]any{1}, args...) // writes into buf -> [1]
out2, _ := parsed([]any{2}, args...) // reuses the same buf -> now [2]
fmt.Println(out1)
fmt.Println(out2)
fmt.Println(buf)
  // Output:
  // [2]
  // [2]
  // [2]

πŸ“ Example

Note: Do not share the same buffer across goroutines concurrently.

* Error handling

If an error occurs during API execution, a specific error type is returned. The following error types help you identify the cause:

Syntax errors from Retrieve or Parse

Error type Message format Symptom Ex
ErrorInvalidSyntax invalid syntax (position=%d, reason=%s, near=%s) The JSONPath contains invalid syntax. The reason in the message provides more details. πŸ“
ErrorInvalidArgument invalid argument (argument=%s, error=%s) An argument in the JSONPath is invalid according to Go syntax. πŸ“
ErrorFunctionNotFound function not found (path=%s) The specified function in the JSONPath was not found. πŸ“
ErrorNotSupported not supported (path=%s, feature=%s) The JSONPath uses unsupported syntax. πŸ“

Runtime errors from Retrieve or parser functions

Error type Message format Symptom Ex
ErrorMemberNotExist member did not exist (path=%s) The specified object or array member does not exist in the JSON object. πŸ“
ErrorTypeUnmatched type unmatched (path=%s, expected=%s, found=%s) The type of the node in the JSON object does not match what is expected by the JSONPath. πŸ“
ErrorFunctionFailed function failed (path=%s, error=%s) The function specified in the JSONPath failed to execute. πŸ“

Type checking makes it easy to determine which error occurred.

import jsonpath "github.com/AsaiYusuke/jsonpath/v2"
import errors "github.com/AsaiYusuke/jsonpath/v2/errors"

_, err := jsonpath.Retrieve(jsonPath, srcJSON)
switch err.(type) {
case errors.ErrorMemberNotExist:
  fmt.Printf("retry with other srcJSON: %v", err)
  // handle or continue
case errors.ErrorInvalidArgument:
  return nil, fmt.Errorf("specified invalid argument: %v", err)
}

* Function syntax

You can use user-defined functions to format results. The function syntax is appended after the JSONPath expression.

There are two types of functions:

Filter function

A filter function applies a user-defined function to each value in the result, transforming them individually.

πŸ“ Example

Aggregate function

An aggregate function combines all values in the result into a single value.

πŸ“ Example

* Accessing JSON

Instead of retrieving values directly, you can obtain accessors (Getters / Setters) for the input JSON. These accessors allow you to update the original JSON object.

Enable this feature by calling Config.SetAccessorMode().

πŸ“ Example

Accessor limitations

Setters are not available for some results, such as when using function syntax in the JSONPath.

Accessor operations follow Go's map/slice semantics. If you modify the structure of the JSON, be aware that accessors may not behave as expected. To avoid issues, obtain a new accessor each time you change the structure.

Differences

Some behaviors in this library differ from the consensus of other implementations. For a full comparison, see πŸ“ this result.

These behaviors may change in the future if more appropriate approaches are found.

Character types

The following character types are allowed for identifiers in dot-child notation:

Character type Available Escape required
* Numbers and alphabets (0-9 A-Z a-z) Yes No
* Hyphen and underscore (- _) Yes No
* Non-ASCII Unicode characters (0x80 - 0x10FFFF) Yes No
* Other printable symbols (Space ! " # $ % & ' ( ) * + , . / : ; < = > ? @ [ \ ] ^ ` { | } ~) Yes Yes
* Control code characters (0x00 - 0x1F, 0x7F) No -

Printable symbols (except hyphen and underscore) can be used by escaping them.

JSONPath : $.abc\.def
srcJSON  : {"abc.def":1}
Output   : [1]

Wildcard in qualifier

Wildcards in qualifiers can be specified as a union with subscripts.

JSONPath : $[0,1:3,*]
srcJSON  : [0,1,2,3,4,5]
Output   : [0,1,2,0,1,2,3,4,5]

Regular expression

Regular expression syntax follows Go's regular expression rules. In particular, you can use "(?i)" to make the regular expression case-insensitive.

JSONPath : $[?(@=~/(?i)CASE/)]
srcJSON  : ["Case","Hello"]
Output   : ["Case"]

JSONPaths in the filter-qualifier

JSONPaths that return a value group cannot be used with a comparator or regular expression. However, you can use these syntaxes for existence checks.

JSONPaths that return a value group example
Recursive descent @..a
Multiple identifier @['a','b']
Wildcard identifier @.*
Slice qualifier @[0:1]
Wildcard qualifier @[*]
Union in the qualifier @[0,1]
Filter qualifier @.a[?(@.b)]
  • comparator example (error)
JSONPath : $[?(@..x == "hello world")]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Error    : ErrorInvalidSyntax
  • regular expression example (error)
JSONPath : $[?(@..x=~/hello/)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Error    : ErrorInvalidSyntax
  • existence check example
JSONPath : $[?(@..x)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Output   : [{"b":{"x":"hello world"}}]

If a JSONPath filter begins with the Root, it performs a whole-match operation if any match is found.

JSONPath : $[?($..x)]
srcJSON  : [{"a":1},{"b":{"x":"hello world"}}]
Output   : [{"a":1},{"b":{"x":"hello world"}}]

Benchmarks

Benchmark results for various Go JSONPath libraries (measured by myself) are available in the following repository:

Project progress

  • Syntax
    • Identifier
      • identifier in dot notation
      • identifier in bracket notation
      • wildcard
      • multiple identifiers in brackets
      • recursive retrieval
    • Qualifier
      • index
      • slice
      • wildcard
      • Filter
        • logical operations
        • comparators
        • JSONPath retrieval in filter
      • script
    • Function
      • filter
      • aggregate
    • Refer to the consensus behaviors
  • Architecture
    • PEG syntax analysis
    • Error handling
    • Function support
    • JSON accessors
  • Go language manner
    • retrieve with an object unmarshaled to interface
    • retrieve with the json.Number type
  • Source code
    • Release version
    • Unit tests
      • syntax tests
      • benchmarks
      • coverage >80%
    • Examples
    • CI automation
    • Documentation
      • README
      • API documentation
    • comparison results (local)
  • Development status
    • requirements and functional design
      • Decided to follow a standard or reference implementation for JSONPath syntax
    • design-based coding
    • testing
    • documentation
  • Future ToDo
    • Go language affinity
      • retrieve with an object unmarshaled to struct
      • retrieve with struct tags
      • retrieve with user-defined objects

About

A query library for retrieving part of JSON based on JSONPath syntax.

Topics

Resources

License

Stars

Watchers

Forks

Contributors 2

  •  
  •  

Languages