jq: Merge Two JSON Files by Key, Tested on 3 Versions
“Merge two JSON files by key” means two different jobs, and jq needs a different filter for each:
- Two objects (a base config and an override), where matching keys should be combined, including nested ones.
- Two arrays of records (users and emails) that share an
idfield, where each record should be joined with its partner, like a SQL join.
I ran every command below against the same small files on three jq builds: the
official 1.8.2 and 1.7.1 release binaries (checksums matched the release’s
sha256sum.txt), and the jq that ships in /usr/bin on macOS 15.3.1, which reports
jq-1.6-159-apple-gcff5336-dirty. All output is copied from the terminal. Where the
three builds disagree, I say so; everywhere else they printed identical output.
Short answer (tested 2026-09-29):
| You want | Use | Watch out for |
|---|---|---|
| Deep-merge two objects, second file wins | jq -s '.[0] * .[1]' base.json override.json | Arrays are replaced, not joined, and a null in the second file overwrites a real value |
| Deep-merge any number of files | jq -n 'reduce inputs as $i ({}; . * $i)' a.json b.json c.json | The start value must be {}; null errors |
Same thing without -s | jq -n 'input as $a | input as $b | $a * $b' | input * input merges in the reverse order |
Join two arrays on id, keep every record | jq -s '(.[0] | INDEX(.id)) * (.[1] | INDEX(.id)) | map(.)' | "1" and 1 end up as the same key |
| Look up extra fields for each record in the first file only | INDEX($e[0][]; .id) as $idx | map(. + ($idx[.id|tostring] // {})) | $idx[.id] fails with a number id; the key must be a string |
| Pass the second file in as a variable | --slurpfile o override.json and then $o[0] | --argfile is gone since 1.7: Unknown option --argfile |
The test files
Two objects:
// base.json
{"name": "app", "server": {"host": "localhost", "port": 8080}, "tags": ["a", "b"], "debug": true}
// override.json
{"server": {"port": 9090}, "tags": ["c"], "debug": null, "extra": 1}
Two arrays of records. Note that the email list is in a different order, has no
record for id 3, and has an extra record for id 4:
// users.json
[{"id": 1, "name": "Ann"}, {"id": 2, "name": "Bob"}, {"id": 3, "name": "Cy"}]
// emails.json
[{"id": 2, "email": "bob@example.com"}, {"id": 1, "email": "ann@example.com"}, {"id": 4, "email": "dee@example.com"}]
Part 1: merging two objects
+ is shallow, * is recursive
$ jq -c -s '.[0] + .[1]' base.json override.json
{"name":"app","server":{"port":9090},"tags":["c"],"debug":null,"extra":1}
$ jq -c -s '.[0] * .[1]' base.json override.json
{"name":"app","server":{"host":"localhost","port":9090},"tags":["c"],"debug":null,"extra":1}
With +, the whole server object from override.json replaced the one from
base.json, so "host":"localhost" is gone. With *, the two server objects were
merged and only port changed. The jq manual describes both: for +, “the object on
the right of the + wins”, and “Multiplying two objects will merge them recursively”.
Two things * does not do, visible in the same output:
- Arrays are replaced, not concatenated.
"tags":["a","b"]became["c"]. nullis a value like any other."debug":truebecame"debug":null, so an override file can’t usenullto mean “leave this alone”.
-s add is the shallow version
A popular answer for merging files is jq -s add. It uses the same rules as +, so
it is a shallow merge:
$ jq -c -s 'add' base.json override.json
{"name":"app","server":{"port":9090},"tags":["c"],"debug":null,"extra":1}
server.host is lost again. With a third file (c.json is
{"server": {"tls": true}, "name": "app2"}), the last file’s server object wipes
out both earlier ones:
$ jq -c -s 'add' base.json override.json c.json
{"name":"app2","server":{"tls":true},"tags":["c"],"debug":null,"extra":1}
jq 1.8 adds add(generator), so jq -n 'add(inputs)' also works without -s.
It is still the + rules and still shallow:
$ jq -c -n 'add(inputs)' base.json override.json
{"name":"app","server":{"port":9090},"tags":["c"],"debug":null,"extra":1}
On 1.7.1 and the macOS build, the same command stops with
jq: error: add/1 is not defined at <top-level>, line 1:.
More than two files: reduce with *
$ jq -c -n 'reduce inputs as $i ({}; . * $i)' base.json override.json c.json
{"name":"app2","server":{"host":"localhost","port":9090,"tls":true},"tags":["c"],"debug":null,"extra":1}
All three server objects were combined, and each later file won on the keys it set.
The start value has to be {}. With null as the start value, all three builds stop
at the first file:
$ jq -c -n 'reduce inputs as $i (null; . * $i)' base.json override.json
jq: error (at base.json:1): null (null) and object ({"name":"app","server":{"...}) cannot be multiplied
(That is 1.8.2’s wording; 1.7.1 and the macOS build cut the object shorter, as
({"name":"ap...).)
The input * input trap: it merges backwards
If you’d rather not load both files into memory with -s, the obvious shorthand is
input * input. The result is the reverse of what you’d expect:
$ jq -c -n 'input * input' base.json override.json
{"server":{"port":8080,"host":"localhost"},"tags":["a","b"],"debug":true,"extra":1,"name":"app"}
port is back to 8080, tags to ["a","b"] and debug to true: base.json
won every conflict, even though it is the first file. The key order gives it away:
the output starts with server, the first key of override.json, so
override.json ended up on the left of *.
On 1.7.1 and 1.8.2, debug(msg) shows why. The right-hand input runs first and
takes the first file:
$ jq -c -n '(input | debug("left")) * (input | debug("right"))' base.json override.json
["DEBUG:","right"]
["DEBUG:","left"]
{"server":{"port":8080,"host":"localhost"},"tags":["a","b"],"debug":true,"extra":1,"name":"app"}
The macOS build has no debug/1, but it printed the same backwards merge for
input * input. I didn’t find evaluation order for * described in the 1.8 manual,
so don’t rely on either order. Bind each file to a variable first, and the order is
the one you wrote:
$ jq -c -n 'input as $a | input as $b | $a * $b' base.json override.json
{"name":"app","server":{"host":"localhost","port":9090},"tags":["c"],"debug":null,"extra":1}
Be careful with the half-way version, input as $b | input | . * $b. It looks like
it reads the second file into $b, but $b gets the first file, and the result is
the same backwards merge as input * input.
--argfile no longer exists
Older answers pass the second file with --argfile. That option was removed in 1.7
(the 1.7 release notes: “Remove deprecated --argfile option”). The macOS build
still accepts it, which is how scripts written on a Mac break on a newer jq:
$ jq -c --argfile o override.json '. * $o' base.json
jq: Unknown option --argfile
Use jq --help for help with command-line options,
or see the jq manpage, or online docs at https://jqlang.org
That is 1.8.2, which exits with code 2. 1.7.1 also exits with code 2 and the same
Unknown option --argfile error. Use --slurpfile. It binds an array of every JSON value in the file,
so take the first one with $o[0]:
$ jq -c --slurpfile o override.json '. * $o[0]' base.json
{"name":"app","server":{"host":"localhost","port":9090},"tags":["c"],"debug":null,"extra":1}
When null should mean “keep” and arrays should add up
* can’t do either, and deleting nulls afterwards is not the same thing:
$ jq -c -s '.[0] * .[1] | del(..|nulls)' base.json override.json
{"name":"app","server":{"host":"localhost","port":9090},"tags":["c"],"extra":1}
debug is now missing entirely instead of keeping true from base.json, and any
null that was in base.json on purpose would be deleted too. What worked on all
three builds was a small recursive function that skips null on the right and
concatenates arrays:
$ jq -c -s 'def deepmerge(a; b): a as $a | b as $b | if ($a|type) == "object" and ($b|type) == "object" then reduce ($b|keys_unsorted[]) as $k ($a; .[$k] = deepmerge($a[$k]; $b[$k])) elif ($a|type) == "array" and ($b|type) == "array" then $a + $b elif $b == null then $a else $b end; deepmerge(.[0]; .[1])' base.json override.json
{"name":"app","server":{"host":"localhost","port":9090},"tags":["a","b","c"],"debug":true,"extra":1}
tags is now ["a","b","c"] and debug stayed true. If you only want one of the
two changes, delete the array branch or the null branch.
Part 2: joining two arrays of records on a key
* and + don’t join arrays
* refuses two arrays outright, and + just puts one after the other:
$ jq -c -s '.[0] * .[1]' users.json emails.json
jq: error (at emails.json:1): array ([{"id":1,"name":"Ann"},{"...]) and array ([{"id":2,"email":"bob@exa...]) cannot be multiplied
$ jq -c -s '.[0] + .[1]' users.json emails.json
[{"id":1,"name":"Ann"},{"id":2,"name":"Bob"},{"id":3,"name":"Cy"},{"id":2,"email":"bob@example.com"},{"id":1,"email":"ann@example.com"},{"id":4,"email":"dee@example.com"}]
(The error text is 1.8.2’s; 1.7.1 and the macOS build cut each array shorter, as
([{"id":1,"n...).)
To join on id, you need one of the three filters below. They differ in which
records they keep and in what order.
Option 1: group_by and add (keeps everything, sorts by id)
$ jq -c -s '.[0] + .[1] | group_by(.id) | map(add)' users.json emails.json
[{"id":1,"name":"Ann","email":"ann@example.com"},{"id":2,"name":"Bob","email":"bob@example.com"},{"id":3,"name":"Cy"},{"id":4,"email":"dee@example.com"}]
This is a full outer join: id 3 (no email) and id 4 (no user) are both kept. The
output is sorted by id, not in the order of either file, and add merges each
group with +, so it is shallow. For nested records, replace map(add) with
map(reduce .[] as $x ({}; . * $x)); on these flat records it printed the same
result.
Option 2: INDEX and a lookup (keeps the first file’s records and order)
INDEX turns an array into an object keyed by whatever expression you give it:
$ jq -c --slurpfile e emails.json 'INDEX($e[0][]; .id)' users.json
{"2":{"id":2,"email":"bob@example.com"},"1":{"id":1,"email":"ann@example.com"},"4":{"id":4,"email":"dee@example.com"}}
The keys are strings, even though id is a number in the file. So look records
up with .id|tostring:
$ jq -c --slurpfile e emails.json 'INDEX($e[0][]; .id) as $idx | map(. + ($idx[.id|tostring] // {}))' users.json
[{"id":1,"name":"Ann","email":"ann@example.com"},{"id":2,"name":"Bob","email":"bob@example.com"},{"id":3,"name":"Cy"}]
This is a left join: every user in the original order, Cy without an email, and
id 4 dropped because there is no such user. The // {} is optional: I ran the
same filter without it and Cy was still kept, because adding null to an object
leaves the object unchanged. It only makes the “no match” case explicit. Without
tostring, all three builds stop on the first record:
$ jq -c --slurpfile e emails.json 'INDEX($e[0][]; .id) as $idx | map(. + $idx[.id])' users.json
jq: error (at users.json:1): Cannot index object with number (1)
(1.7.1 and the macOS build print the same message without the (1).)
The built-in JOIN does the same lookup but gives you pairs, with null where
nothing matched:
$ jq -c --slurpfile e emails.json '[JOIN(INDEX($e[0][]; .id); .[]; .id|tostring)]' users.json
[[{"id":1,"name":"Ann"},{"id":1,"email":"ann@example.com"}],[{"id":2,"name":"Bob"},{"id":2,"email":"bob@example.com"}],[{"id":3,"name":"Cy"},null]]
Adding add as the fourth argument merges each pair and prints the same result as
the lookup above: [JOIN(INDEX($e[0][]; .id); .[]; .id|tostring; add)].
Option 3: INDEX both files and multiply (keeps everything, keeps order)
$ jq -c -s '(.[0] | INDEX(.id)) * (.[1] | INDEX(.id)) | map(.)' users.json emails.json
[{"id":1,"name":"Ann","email":"ann@example.com"},{"id":2,"name":"Bob","email":"bob@example.com"},{"id":3,"name":"Cy"},{"id":4,"email":"dee@example.com"}]
Both arrays become objects keyed by id, * merges them recursively, and map(.)
turns the object back into an array. It keeps every record, merges nested fields,
and keeps the first file’s order, with records only in the second file added at the
end. id 4 lands last because it is new. In this test, the result happened to match
the sorted group_by version only because users.json was already sorted.
Three things that silently change the result
A string id and a number id. If one file has "id": "1" and the other has
"id": 1, group_by treats them as different values and doesn’t join anything:
$ jq -c -s '.[0] + .[1] | group_by(.id) | map(add)' users_strid.json emails.json
[{"id":1,"email":"ann@example.com"},{"id":2,"email":"bob@example.com"},{"id":4,"email":"dee@example.com"},{"id":"1","name":"Ann"},{"id":"2","name":"Bob"}]
The INDEX versions do join them, because both keys become the string "1". The
merged record keeps the second file’s id, which here means the number 1:
$ jq -c -s '(.[0] | INDEX(.id)) * (.[1] | INDEX(.id)) | map(.)' users_strid.json emails.json
[{"id":1,"name":"Ann","email":"ann@example.com"},{"id":2,"name":"Bob","email":"bob@example.com"},{"id":4,"email":"dee@example.com"}]
If your ids might be mixed, normalize them first. map(map(.id |= tostring)) turns
every id in both arrays into a string before the join:
$ jq -c -s 'map(map(.id |= tostring)) | (.[0] | INDEX(.id)) * (.[1] | INDEX(.id)) | map(.)' users_strid.json emails.json
[{"id":"1","name":"Ann","email":"ann@example.com"},{"id":"2","name":"Bob","email":"bob@example.com"},{"id":"4","email":"dee@example.com"}]
Duplicate ids. INDEX keeps the last record for each key and says nothing.
With dup.json holding two records for id 1, old@example.com then
new@example.com:
$ jq -c -s '(.[0] | INDEX(.id)) * (.[1] | INDEX(.id)) | map(.)' users.json dup.json
[{"id":1,"name":"Ann","email":"new@example.com"},{"id":2,"name":"Bob"},{"id":3,"name":"Cy"}]
Check for duplicates first if losing one would matter:
$ jq -c '[group_by(.id)[] | select(length > 1)]' dup.json
[[{"id":1,"email":"old@example.com"},{"id":1,"email":"new@example.com"}]]
Very large numeric ids. I tested ids above 2^53 (9007199254740992 and
9007199254740993). On all three builds, printing, comparing, tostring,
group_by and INDEX kept the two ids apart and joined correctly. Arithmetic did
not: map(.id + 0) printed [9007199254740992,9007199254740992]. So don’t do math
on an id before using it as a join key. I did not test the upstream jq 1.6 release
that some Linux distributions ship, only Apple’s 1.6 build.
What differed between versions
macOS /usr/bin/jq (1.6-apple) | 1.7.1 | 1.8.2 | |
|---|---|---|---|
*, +, -s add, reduce, group_by, INDEX, JOIN | same output | same output | same output |
--argfile | works | Unknown option --argfile | Unknown option --argfile |
add(generator) | not defined | not defined | works |
pick(path) | not defined | works | works |
debug(msg) | not defined | works | works |
The 1.7 and 1.8 release notes list the same changes: --argfile removed and
pick added in 1.7, add/1 added in 1.8.0. So if a script has to run on a Mac’s
built-in jq and on a current Linux jq, stick to the first row of the table and use
--slurpfile.
Which one to use
- Config files:
jq -s '.[0] * .[1]'for two files,reduce inputs as $i ({}; . * $i)for more. Use thedeepmergefunction only if you neednullto mean “keep” or arrays to be concatenated. - Records by id, keep everything: INDEX both files and multiply. It keeps the
order of the first file and merges nested fields, which
group_by+adddoes not. - Records by id, first file decides which records exist: the
INDEXlookup with.id|tostringand// {}. - Never
input * input, and never--argfilein anything that might run on jq 1.7 or later.
The manual sections quoted above are in the official jq manual: https://jqlang.org/manual/