CERT Airs Flaws in OpenSSL Protocol

The software engineering watchdog discloses some nasty holes in the OpenSSL security software, an open-source implementation of security protocols used to protect many online transactions.

by Clint Boulton of internetnews.com [August 1, 2002]

Security experts have detected four serious vulnerabilities in OpenSSL, an open source implementation of the Secure Sockets Layer (SSL v2/v3) and its successor, Transport Layer Security (TLS v1), according to the Computer Emergency Response Team (CERT).

The flaws, commonly known as buffer overflows, could be used by a malicious perpetrator to execute code on a system, or simply to inflict denial-of-service attacks. Versions OpenSSL that CERT considers susceptible to a breach include OpenSSL prior to 0.9.6e, up to and including pre-release 0.9.7-beta2, as well as OpenSSL pre-release 0.9.7-beta2 and prior with Kerberos-enabled SSLeay library.

SSL and TLS protocols are used to provide a secure connection between a client and a server for higher level protocols, such as HTTP. Developed by Netscape, SSL gained the support of Microsoft and other Internet client/server developers and became the de facto standard until evolving into TLS. OpenSSL is an open-source derivative of SSL and TLS. OpenSSL stands apart from the other two in that it also functions as a cryptography library.

The vulnerabilities were enough to make one security company issue a cautionary public statement.

"The wide proliferation of servers that utilize SSL to protect encrypted sessions may make these vulnerabilities significant," said Bruce Murphy, CEO of Vigilinx. "At this point, we are not aware of any damage that has been inflicted due to this vulnerability, but we believe that the potential for damage is high and a proactive response is warranted."

The flaws include:

• VU#102795—OpenSSL servers contain a buffer overflow during the SSLv2 handshake process. This can be exploited by a client using a malformed key during the handshake process with an SSL server connection
• VU#258555—OpenSSL clients contain a buffer overflow during the SSLv3 handshake process. A malicious server can exploit this by sending a large session ID to the client during the handshake process.
• VU#561275—OpenSSL servers with Kerberos enabled contain an exploitable buffer overflow during the SSLv3 handshake process. Users may exploit this if a malicious client sends a malformed key during the SSLv3 handshake process with the server
• VU#308891—OpenSSL contains multiple buffers overflows in buffers that are used to hold ASCII representations of integers

CERT also warned of holes involving malformed ASN.1 encodings in OpenSSL. Those affected include SSL or TLS applications, as well as S/MIME, PKCS#7, and certificate creation routines. Wit this flaw, the ASN.1 library has various encoding errors that allow malformed certificate encodings to be parsed incorrectly, leaving it open to denial-of-service issues.

CERT recommends that those affected by the flaws upgrade to version 0.9.6e of OpenSSL. Patches for the flaw are available from OpenSSL.org are available: For the OpenSSL 0.9.6d patch go here.

Sites running OpenSSL pre-release version 0.9.7-beta2 may wish to upgrade to 0.9.7-beta3. Combined patches for OpenSSL 0.9.7 beta 2 may be found here.

After either applying the patches above or upgrading to 0.9.6e, CERT advises users to recompile all applications using OpenSSL to support SSL or TLS services, and restart services or systems to punt vulnerable code.

CERT was made aware of the holes by various people. A.L. Digital Ltd and John McDonald of Neohapsis discovered VU#102795. A.L. Digital Ltd discovered VU#258555, VU#561275, and VU#308891. Adi Stav and James Yonan each independently discovered VU#748355.

— End