Improper calculations in ECC implementation can trigger a Denial-of-Service (DoS)

Description

When using the non-default "fallback" crypto back-end, ECC operations in node-jose can trigger a Denial-of-Service (DoS) condition, due to a possible infinite loop in an internal calculation. For some ECC operations, this condition is triggered randomly; for others, it can be triggered by malicious input.

Technical summary

The JOSE logic implemented by node-jose usually relies on an external cryptographic library for the underlying cryptographic primitives that JOSE operations require. When WebCrypto or the Node crypto module are available, they are used. When neither of these libraries is available, node-jose includes its own "fallback" implementations of some algorithms based on node-forge, in particular implementations of ECDH and ECDSA.

A various points, these algorithm implementations need to compute to the X coordinate of an elliptic curve point. This is done by calling the getX() method of the object representing the point, which is an alias of the function pointFpGetX() in lib/deps/ecc/math.js.

Computing the X coordinate from the form in which the point is stored requires computing the modular inverse of the Z coordinate, using the modInverse function from the jsbn library (e.g., this.z.modInverse(this.curve.p)). The output of this function call is multiplied by another value before being reduced with the barrettReduce() function.

The root cause of this issue is that the jsbn modInverse function sometimes returns negative results. These results are correct in that they are equivalent mod the relevant modulus, but can be problematic for functions that expect modular operations to always return positive results (in the range [0, p), where p is the modulus).

In particular, while the Barrett reduction algorithm in general can handle negative inputs, the implementation in node-jose explicitly does not. Therefore, while the negative value that is returned by modInverse() is mathematically correct, it leads to an error in barrettReduce() causing an infinite loop which may result in a Denial of Service condition.

For a given prime modulus, we estimate that roughly one in every 2^20 inputs produce a negative modInverse(). This estimate was validated with exhaustive testing on small primes (<30 bits) and randomized testing with regard to the P-256 prime.

Impact

This issue is only present in situations where the "fallback" cryptographic implementation is being used, i.e., situations where neither WebCrypto nor the Node module is available.