1 /****************************************************************************
3 ** Copyright (C) 2018 Intel Corporation
5 ** Permission is hereby granted, free of charge, to any person obtaining a copy
6 ** of this software and associated documentation files (the "Software"), to deal
7 ** in the Software without restriction, including without limitation the rights
8 ** to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 ** copies of the Software, and to permit persons to whom the Software is
10 ** furnished to do so, subject to the following conditions:
12 ** The above copyright notice and this permission notice shall be included in
13 ** all copies or substantial portions of the Software.
15 ** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 ** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 ** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 ** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 ** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 ** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
23 ****************************************************************************/
26 #define _DEFAULT_SOURCE 1
28 #define _POSIX_C_SOURCE 200809L
29 #ifndef __STDC_LIMIT_MACROS
30 # define __STDC_LIMIT_MACROS 1
35 #include "cborinternal_p.h"
36 #include "compilersupport_p.h"
44 * \defgroup CborToJson Converting CBOR to JSON
45 * \brief Group of functions used to convert CBOR to JSON.
47 * This group contains two functions that can be used to convert a \ref
48 * CborValue object to an equivalent JSON representation. This module attempts
49 * to follow the recommendations from RFC 7049 section 4.1 "Converting from
50 * CBOR to JSON", though it has a few differences. They are noted below.
52 * These functions produce a "minified" JSON output, with no spacing,
53 * indentation or line breaks. If those are necessary, they need to be applied
54 * in a post-processing phase.
56 * Note that JSON cannot support all CBOR types with fidelity, so the
57 * conversion is usually lossy. For that reason, TinyCBOR supports adding a set
58 * of metadata JSON values that can be used by a JSON-to-CBOR converter to
59 * restore the original data types.
61 * The TinyCBOR library does not provide a way to convert from JSON
62 * representation back to encoded form. However, it provides a tool called
63 * \c json2cbor which can be used for that purpose. That tool supports the
64 * metadata format that these functions may produce.
66 * Either of the functions in this section will attempt to convert exactly one
67 * CborValue object to JSON. Those functions may return any error documented
68 * for the functions for CborParsing. In addition, if the C standard library
69 * stream functions return with error, the text conversion will return with
72 * These functions also perform UTF-8 validation in CBOR text strings. If they
73 * encounter a sequence of bytes that is not permitted in UTF-8, they will return
74 * CborErrorInvalidUtf8TextString. That includes encoding of surrogate points
77 * \warning The metadata produced by these functions is not guaranteed to
78 * remain stable. A future update of TinyCBOR may produce different output for
79 * the same input and parsers may be unable to handle it.
81 * \sa CborParsing, CborPretty, cbor_parser_init()
85 * \addtogroup CborToJson
87 * <h2 class="groupheader">Conversion limitations</h2>
89 * When converting from CBOR to JSON, there may be information loss. This
90 * section lists the possible scenarios.
92 * \par Number precision:
93 * ALL JSON numbers, due to its JavaScript heritage, are IEEE 754
94 * double-precision floating point. This means JSON is not capable of
95 * representing all integers numbers outside the range [-(2<sup>53</sup>)+1,
96 * 2<sup>53</sup>-1] and is not capable of representing NaN or infinite. If the
97 * CBOR data contains a number outside the valid range, the conversion will
98 * lose precision. If the input was NaN or infinite, the result of the
99 * conversion will be the JSON null value. In addition, the distinction between
100 * half-, single- and double-precision is lost.
103 * If enabled, the original value and original type are stored in the metadata.
105 * \par Non-native types:
106 * CBOR's type system is richer than JSON's, which means some data values
107 * cannot be represented when converted to JSON. The conversion silently turns
108 * them into strings: CBOR simple types become "simple(nn)" where \c nn is the
109 * simple type's value, with the exception of CBOR undefined, which becomes
110 * "undefined", while CBOR byte strings are converted to an Base16, Base64, or
114 * If enabled, the original type is stored in the metadata.
116 * \par Presence of tags:
117 * JSON has no support for tagged values, so by default tags are dropped when
118 * converting to JSON. However, if the CborConvertObeyByteStringTags option is
119 * active (default), then certain known tags are honored and are used to format
120 * the conversion of the tagged byte string to JSON.
123 * If the CborConvertTagsToObjects option is active, then the tag and the
124 * tagged value are converted to a JSON object. Otherwise, if enabled, the
125 * last (innermost) tag is stored in the metadata.
127 * \par Non-string keys in maps:
128 * JSON requires all Object keys to be strings, while CBOR does not. By
129 * default, if a non-string key is found, the conversion fails with error
130 * CborErrorJsonObjectKeyNotString. If the CborConvertStringifyMapKeys option
131 * is active, then the conversion attempts to create a string representation
132 * using CborPretty. Note that the \c json2cbor tool is not able to parse this
133 * back to the original form.
135 * \par Duplicate keys in maps:
136 * Neither JSON nor CBOR allow duplicated keys, but current TinyCBOR does not
137 * validate that this is the case. If there are duplicated keys in the input,
138 * they will be repeated in the output, which many JSON tools may flag as
139 * invalid. In addition to that, if the CborConvertStringifyMapKeys option is
140 * active, it is possible that a non-string key in a CBOR map will be converted
141 * to a string form that is identical to another key.
144 * When metadata support is active, the conversion will add extra key-value
145 * pairs to the JSON output so it can store the metadata. It is possible that
146 * the keys for the metadata clash with existing keys in the JSON map.
149 extern FILE *open_memstream(char **bufptr
, size_t *sizeptr
);
151 enum ConversionStatusFlags
{
152 TypeWasNotNative
= 0x100, /* anything but strings, boolean, null, arrays and maps */
153 TypeWasTagged
= 0x200,
154 NumberPrecisionWasLost
= 0x400,
155 NumberWasNaN
= 0x800,
156 NumberWasInfinite
= 0x1000,
157 NumberWasNegative
= 0x2000, /* only used with NumberWasInifite or NumberWasTooBig */
162 typedef struct ConversionStatus
{
164 uint64_t originalNumber
;
168 static CborError
value_to_json(FILE *out
, CborValue
*it
, int flags
, CborType type
, ConversionStatus
*status
);
170 static CborError
dump_bytestring_base16(char **result
, CborValue
*it
)
172 static const char characters
[] = "0123456789abcdef";
176 CborError err
= cbor_value_calculate_string_length(it
, &n
);
180 /* a Base16 (hex) output is twice as big as our buffer */
181 buffer
= (uint8_t *)malloc(n
* 2 + 1);
182 *result
= (char *)buffer
;
184 /* let cbor_value_copy_byte_string know we have an extra byte for the terminating NUL */
186 err
= cbor_value_copy_byte_string(it
, buffer
+ n
- 1, &n
, it
);
187 cbor_assert(err
== CborNoError
);
189 for (i
= 0; i
< n
; ++i
) {
190 uint8_t byte
= buffer
[n
+ i
];
191 buffer
[2*i
] = characters
[byte
>> 4];
192 buffer
[2*i
+ 1] = characters
[byte
& 0xf];
197 static CborError
generic_dump_base64(char **result
, CborValue
*it
, const char alphabet
[65])
200 uint8_t *buffer
, *out
, *in
;
201 CborError err
= cbor_value_calculate_string_length(it
, &n
);
205 /* a Base64 output (untruncated) has 4 bytes for every 3 in the input */
206 size_t len
= (n
+ 5) / 3 * 4;
207 out
= buffer
= (uint8_t *)malloc(len
+ 1);
208 *result
= (char *)buffer
;
210 /* we read our byte string at the tail end of the buffer
211 * so we can do an in-place conversion while iterating forwards */
212 in
= buffer
+ len
- n
;
214 /* let cbor_value_copy_byte_string know we have an extra byte for the terminating NUL */
216 err
= cbor_value_copy_byte_string(it
, in
, &n
, it
);
217 cbor_assert(err
== CborNoError
);
219 uint_least32_t val
= 0;
220 for (i
= 0; n
- i
>= 3; i
+= 3) {
221 /* read 3 bytes x 8 bits = 24 bits */
225 __builtin_memcpy(&val
, in
+ i
- 1, sizeof(val
));
226 val
= cbor_ntohl(val
);
229 val
= (in
[i
] << 16) | (in
[i
+ 1] << 8) | in
[i
+ 2];
232 /* write 4 chars x 6 bits = 24 bits */
233 *out
++ = alphabet
[(val
>> 18) & 0x3f];
234 *out
++ = alphabet
[(val
>> 12) & 0x3f];
235 *out
++ = alphabet
[(val
>> 6) & 0x3f];
236 *out
++ = alphabet
[val
& 0x3f];
239 /* maybe 1 or 2 bytes left */
241 /* we can read in[i + 1] even if it's past the end of the string because
242 * we know (by construction) that it's a NUL byte */
245 __builtin_memcpy(&val16
, in
+ i
, sizeof(val16
));
246 val
= cbor_ntohs(val16
);
248 val
= (in
[i
] << 8) | in
[i
+ 1];
252 /* the 65th character in the alphabet is our filler: either '=' or '\0' */
254 out
[3] = alphabet
[64];
256 /* write the third char in 3 chars x 6 bits = 18 bits */
257 out
[2] = alphabet
[(val
>> 6) & 0x3f];
259 out
[2] = alphabet
[64]; /* filler */
261 out
[1] = alphabet
[(val
>> 12) & 0x3f];
262 out
[0] = alphabet
[(val
>> 18) & 0x3f];
270 static CborError
dump_bytestring_base64(char **result
, CborValue
*it
)
272 static const char alphabet
[] = "ABCDEFGH" "IJKLMNOP" "QRSTUVWX" "YZabcdef"
273 "ghijklmn" "opqrstuv" "wxyz0123" "456789+/" "=";
274 return generic_dump_base64(result
, it
, alphabet
);
277 static CborError
dump_bytestring_base64url(char **result
, CborValue
*it
)
279 static const char alphabet
[] = "ABCDEFGH" "IJKLMNOP" "QRSTUVWX" "YZabcdef"
280 "ghijklmn" "opqrstuv" "wxyz0123" "456789-_";
281 return generic_dump_base64(result
, it
, alphabet
);
284 static CborError
add_value_metadata(FILE *out
, CborType type
, const ConversionStatus
*status
)
286 int flags
= status
->flags
;
287 if (flags
& TypeWasTagged
) {
288 /* extract the tagged type, which may be JSON native */
289 type
= flags
& FinalTypeMask
;
290 flags
&= ~(FinalTypeMask
| TypeWasTagged
);
292 if (fprintf(out
, "\"tag\":\"%" PRIu64
"\"%s", status
->lastTag
,
293 flags
& ~TypeWasTagged
? "," : "") < 0)
300 /* print at least the type */
301 if (fprintf(out
, "\"t\":%d", type
) < 0)
304 if (flags
& NumberWasNaN
)
305 if (fprintf(out
, ",\"v\":\"nan\"") < 0)
307 if (flags
& NumberWasInfinite
)
308 if (fprintf(out
, ",\"v\":\"%sinf\"", flags
& NumberWasNegative
? "-" : "") < 0)
310 if (flags
& NumberPrecisionWasLost
)
311 if (fprintf(out
, ",\"v\":\"%c%" PRIx64
"\"", flags
& NumberWasNegative
? '-' : '+',
312 status
->originalNumber
) < 0)
314 if (type
== CborSimpleType
)
315 if (fprintf(out
, ",\"v\":%d", (int)status
->originalNumber
) < 0)
320 static CborError
find_tagged_type(CborValue
*it
, CborTag
*tag
, CborType
*type
)
322 CborError err
= CborNoError
;
323 *type
= cbor_value_get_type(it
);
324 while (*type
== CborTagType
) {
325 cbor_value_get_tag(it
, tag
); /* can't fail */
326 err
= cbor_value_advance_fixed(it
);
330 *type
= cbor_value_get_type(it
);
335 static CborError
tagged_value_to_json(FILE *out
, CborValue
*it
, int flags
, ConversionStatus
*status
)
340 if (flags
& CborConvertTagsToObjects
) {
341 cbor_value_get_tag(it
, &tag
); /* can't fail */
342 err
= cbor_value_advance_fixed(it
);
346 if (fprintf(out
, "{\"tag%" PRIu64
"\":", tag
) < 0)
349 CborType type
= cbor_value_get_type(it
);
350 err
= value_to_json(out
, it
, flags
, type
, status
);
353 if (flags
& CborConvertAddMetadata
&& status
->flags
) {
354 if (fprintf(out
, ",\"tag%" PRIu64
"$cbor\":{", tag
) < 0 ||
355 add_value_metadata(out
, type
, status
) != CborNoError
||
359 if (fputc('}', out
) < 0)
361 status
->flags
= TypeWasNotNative
| CborTagType
;
366 err
= find_tagged_type(it
, &status
->lastTag
, &type
);
369 tag
= status
->lastTag
;
371 /* special handling of byte strings? */
372 if (type
== CborByteStringType
&& (flags
& CborConvertByteStringsToBase64Url
) == 0 &&
373 (tag
== CborNegativeBignumTag
|| tag
== CborExpectedBase16Tag
|| tag
== CborExpectedBase64Tag
)) {
377 if (tag
== CborNegativeBignumTag
) {
379 err
= dump_bytestring_base64url(&str
, it
);
380 } else if (tag
== CborExpectedBase64Tag
) {
381 err
= dump_bytestring_base64(&str
, it
);
382 } else { /* tag == CborExpectedBase16Tag */
383 err
= dump_bytestring_base16(&str
, it
);
387 err
= fprintf(out
, "\"%s%s\"", pre
, str
) < 0 ? CborErrorIO
: CborNoError
;
389 status
->flags
= TypeWasNotNative
| TypeWasTagged
| CborByteStringType
;
393 /* no special handling */
394 err
= value_to_json(out
, it
, flags
, type
, status
);
395 status
->flags
|= TypeWasTagged
| type
;
399 static CborError
stringify_map_key(char **key
, CborValue
*it
, int flags
, CborType type
)
401 (void)flags
; /* unused */
402 (void)type
; /* unused */
403 #ifdef WITHOUT_OPEN_MEMSTREAM
404 (void)key
; /* unused */
405 (void)it
; /* unused */
406 return CborErrorJsonNotImplemented
;
410 FILE *memstream
= open_memstream(key
, &size
);
411 if (memstream
== NULL
)
412 return CborErrorOutOfMemory
; /* could also be EMFILE, but it's unlikely */
413 CborError err
= cbor_value_to_pretty_advance(memstream
, it
);
415 if (unlikely(fclose(memstream
) < 0 || *key
== NULL
))
416 return CborErrorInternalError
;
421 static CborError
array_to_json(FILE *out
, CborValue
*it
, int flags
, ConversionStatus
*status
)
423 const char *comma
= "";
424 while (!cbor_value_at_end(it
)) {
425 if (fprintf(out
, "%s", comma
) < 0)
429 CborError err
= value_to_json(out
, it
, flags
, cbor_value_get_type(it
), status
);
436 static CborError
map_to_json(FILE *out
, CborValue
*it
, int flags
, ConversionStatus
*status
)
438 const char *comma
= "";
440 while (!cbor_value_at_end(it
)) {
442 if (fprintf(out
, "%s", comma
) < 0)
446 CborType keyType
= cbor_value_get_type(it
);
447 if (likely(keyType
== CborTextStringType
)) {
449 err
= cbor_value_dup_text_string(it
, &key
, &n
, it
);
450 } else if (flags
& CborConvertStringifyMapKeys
) {
451 err
= stringify_map_key(&key
, it
, flags
, keyType
);
453 return CborErrorJsonObjectKeyNotString
;
458 /* first, print the key */
459 if (fprintf(out
, "\"%s\":", key
) < 0) {
464 /* then, print the value */
465 CborType valueType
= cbor_value_get_type(it
);
466 err
= value_to_json(out
, it
, flags
, valueType
, status
);
468 /* finally, print any metadata we may have */
469 if (flags
& CborConvertAddMetadata
) {
470 if (!err
&& keyType
!= CborTextStringType
) {
471 if (fprintf(out
, ",\"%s$keycbordump\":true", key
) < 0)
474 if (!err
&& status
->flags
) {
475 if (fprintf(out
, ",\"%s$cbor\":{", key
) < 0 ||
476 add_value_metadata(out
, valueType
, status
) != CborNoError
||
489 static CborError
value_to_json(FILE *out
, CborValue
*it
, int flags
, CborType type
, ConversionStatus
*status
)
499 err
= cbor_value_enter_container(it
, &recursed
);
501 it
->ptr
= recursed
.ptr
;
502 return err
; /* parse error */
504 if (fputc(type
== CborArrayType
? '[' : '{', out
) < 0)
507 err
= (type
== CborArrayType
) ?
508 array_to_json(out
, &recursed
, flags
, status
) :
509 map_to_json(out
, &recursed
, flags
, status
);
511 it
->ptr
= recursed
.ptr
;
512 return err
; /* parse error */
515 if (fputc(type
== CborArrayType
? ']' : '}', out
) < 0)
517 err
= cbor_value_leave_container(it
, &recursed
);
519 return err
; /* parse error */
521 status
->flags
= 0; /* reset, there are never conversion errors for us */
525 case CborIntegerType
: {
526 double num
; /* JS numbers are IEEE double precision */
528 cbor_value_get_raw_integer(it
, &val
); /* can't fail */
531 if (cbor_value_is_negative_integer(it
)) {
532 num
= -num
- 1; /* convert to negative */
533 if ((uint64_t)(-num
- 1) != val
) {
534 status
->flags
= NumberPrecisionWasLost
| NumberWasNegative
;
535 status
->originalNumber
= val
;
538 if ((uint64_t)num
!= val
) {
539 status
->flags
= NumberPrecisionWasLost
;
540 status
->originalNumber
= val
;
543 if (fprintf(out
, "%.0f", num
) < 0) /* this number has no fraction, so no decimal points please */
548 case CborByteStringType
:
549 case CborTextStringType
: {
551 if (type
== CborByteStringType
) {
552 err
= dump_bytestring_base64url(&str
, it
);
553 status
->flags
= TypeWasNotNative
;
556 err
= cbor_value_dup_text_string(it
, &str
, &n
, it
);
560 err
= (fprintf(out
, "\"%s\"", str
) < 0) ? CborErrorIO
: CborNoError
;
566 return tagged_value_to_json(out
, it
, flags
, status
);
568 case CborSimpleType
: {
570 cbor_value_get_simple_type(it
, &simple_type
); /* can't fail */
571 status
->flags
= TypeWasNotNative
;
572 status
->originalNumber
= simple_type
;
573 if (fprintf(out
, "\"simple(%" PRIu8
")\"", simple_type
) < 0)
579 if (fprintf(out
, "null") < 0)
583 case CborUndefinedType
:
584 status
->flags
= TypeWasNotNative
;
585 if (fprintf(out
, "\"undefined\"") < 0)
589 case CborBooleanType
: {
591 cbor_value_get_boolean(it
, &val
); /* can't fail */
592 if (fprintf(out
, val
? "true" : "false") < 0)
597 #ifndef CBOR_NO_FLOATING_POINT
598 case CborDoubleType
: {
603 status
->flags
= TypeWasNotNative
;
604 cbor_value_get_float(it
, &f
);
608 case CborHalfFloatType
:
609 # ifndef CBOR_NO_HALF_FLOAT_TYPE
610 status
->flags
= TypeWasNotNative
;
611 cbor_value_get_half_float(it
, &f16
);
612 val
= decode_half(f16
);
615 err
= CborErrorUnsupportedType
;
619 cbor_value_get_double(it
, &val
);
622 int r
= fpclassify(val
);
623 if (r
== FP_NAN
|| r
== FP_INFINITE
) {
624 if (fprintf(out
, "null") < 0)
626 status
->flags
|= r
== FP_NAN
? NumberWasNaN
:
627 NumberWasInfinite
| (val
< 0 ? NumberWasNegative
: 0);
629 uint64_t ival
= (uint64_t)fabs(val
);
630 if ((double)ival
== fabs(val
)) {
631 /* print as integer so we get the full precision */
632 r
= fprintf(out
, "%s%" PRIu64
, val
< 0 ? "-" : "", ival
);
633 status
->flags
|= TypeWasNotNative
; /* mark this integer number as a double */
635 /* this number is definitely not a 64-bit integer */
636 r
= fprintf(out
, "%." DBL_DECIMAL_DIG_STR
"g", val
);
646 case CborHalfFloatType
:
647 err
= CborErrorUnsupportedType
;
649 #endif /* !CBOR_NO_FLOATING_POINT */
651 case CborInvalidType
:
652 return CborErrorUnknownType
;
655 return cbor_value_advance_fixed(it
);
659 * \enum CborToJsonFlags
660 * The CborToJsonFlags enum contains flags that control the conversion of CBOR to JSON.
662 * \value CborConvertAddMetadata Adds metadata to facilitate restoration of the original CBOR data.
663 * \value CborConvertTagsToObjects Converts CBOR tags to JSON objects
664 * \value CborConvertIgnoreTags (default) Ignore CBOR tags, except for byte strings
665 * \value CborConvertObeyByteStringTags (default) Honor formatting of CBOR byte strings if so tagged
666 * \value CborConvertByteStringsToBase64Url Force the conversion of all CBOR byte strings to Base64url encoding, despite any tags
667 * \value CborConvertRequireMapStringKeys (default) Require CBOR map keys to be strings, failing the conversion if they are not
668 * \value CborConvertStringifyMapKeys Convert non-string keys in CBOR maps to a string form
669 * \value CborConvertDefaultFlags Default conversion flags.
673 * \fn CborError cbor_value_to_json(FILE *out, const CborValue *value, int flags)
675 * Converts the current CBOR type pointed to by \a value to JSON and writes that
676 * to the \a out stream. If an error occurs, this function returns an error
677 * code similar to CborParsing. The \a flags parameter indicates one or more of
678 * the flags from CborToJsonFlags that control the conversion.
680 * \sa cbor_value_to_json_advance(), cbor_value_to_pretty()
684 * Converts the current CBOR type pointed to by \a value to JSON and writes that
685 * to the \a out stream. If an error occurs, this function returns an error
686 * code similar to CborParsing. The \a flags parameter indicates one or more of
687 * the flags from CborToJsonFlags that control the conversion.
689 * If no error ocurred, this function advances \a value to the next element.
691 * \sa cbor_value_to_json(), cbor_value_to_pretty_advance()
693 CborError
cbor_value_to_json_advance(FILE *out
, CborValue
*value
, int flags
)
695 ConversionStatus status
;
696 return value_to_json(out
, value
, flags
, cbor_value_get_type(value
), &status
);