| 1 |
Lynx SSL support for certificates - README.sslcerts file |
| 2 |
|
| 3 |
BACKGROUND: |
| 4 |
|
| 5 |
The original README.ssl document for lynx stated: |
| 6 |
|
| 7 |
Note that the server... may not have a valid certificate. Lynx will not |
| 8 |
complain, as it does not yet support certificates... |
| 9 |
|
| 10 |
Such lack of support is no longer the case. Lynx now features excellent |
| 11 |
certificate management through the openssl project. There is almost no |
| 12 |
online documentation available regarding how to use openssl's certificate |
| 13 |
management with other programs, so this will accompany lynx and hopefully |
| 14 |
encourage good practical security for unix clients. |
| 15 |
|
| 16 |
Lynx relies on openssl to not only encrypt connections over https, but also to |
| 17 |
determine whether it should even accept a certificate and establish a secure |
| 18 |
connection with a remote host. Because of this reliance upon openssl by lynx, |
| 19 |
most of this tutorial deals with how to use openssl to "install" both |
| 20 |
vendor-provided CA cert bundles as well as self-signed certs from trusted sources |
| 21 |
and, most importantly, how to get them recognized by lynx. |
| 22 |
|
| 23 |
While lynx on many systems will transparently accept valid certificates, not |
| 24 |
all systems enjoy such functionality. Further, as noted above, older versions |
| 25 |
of lynx do not perform any validity checks on a certificate. |
| 26 |
|
| 27 |
There is also the common case of wanting to trust, use and install a |
| 28 |
self-signed certificate from a known server source and have it be trusted by |
| 29 |
client programs. |
| 30 |
|
| 31 |
Briefly, the procedure will involve confirming the default system location for |
| 32 |
certificates, setting values for SSL_CERT_DIR and SSL_CERT_FILE in |
| 33 |
the environment, and converting and hashing the certificates using openssl |
| 34 |
utilities to enable recognition. |
| 35 |
|
| 36 |
THE CURRENT SITUATION: |
| 37 |
|
| 38 |
Prior to lynx2.8.5dev9, lynx did not check at all for certificate validity. |
| 39 |
|
| 40 |
Since lynx2.8.5.dev9, lynx has reported this openssl error: |
| 41 |
|
| 42 |
SSL error:unable to get local issuer certificate-Continue? (y) |
| 43 |
|
| 44 |
whenever an https connection was initiated and the certificate could not be |
| 45 |
found, for whatever reason, by openssl, and therefore lynx. |
| 46 |
|
| 47 |
This checking for a certificate is an enhancement to security, but rather |
| 48 |
tediously generates errors at each https browser request. |
| 49 |
|
| 50 |
The ability to turn off reporting of this error to the user was added to |
| 51 |
lynx2.8.5dev16 as the FORCE_SSL_PROMPT setting in lynx.cfg as noted in the |
| 52 |
CHANGELOG: |
| 53 |
|
| 54 |
This lets the user decide whether to ignore prompting for questionable |
| 55 |
aspects of an SSL connection. |
| 56 |
|
| 57 |
While this is a convenient setting to employ when using lynx to script |
| 58 |
https -dumps, it by definition ignores the issue of certificate validity |
| 59 |
altogether. Those concerned with proper certificate management and |
| 60 |
the maintenance of a store of updated CA certificates will be uncomfortable |
| 61 |
with this relaxed security setting. |
| 62 |
|
| 63 |
The ability to accept a 'wildcard' certificate, where the first character |
| 64 |
is a '*' was added to lynx2.8.6dev18. |
| 65 |
|
| 66 |
PRELIMINARY PROCEDURES: |
| 67 |
|
| 68 |
It is assumed that openssl has been installed correctly, that the default |
| 69 |
cert directory is /usr/local/ssl/certs, (it's often /etc/ssl/certs, but we |
| 70 |
need a point of departure for the discussion) and that lynx has been compiled |
| 71 |
--with-ssl. |
| 72 |
|
| 73 |
The default location for certs on your system may be different, or there may not |
| 74 |
be one. You will have to substitute that location for /usr/local/ssl/certs in |
| 75 |
the following instructions, and/or set environment variables. |
| 76 |
|
| 77 |
To determine the default location for certs on your system you may run the |
| 78 |
following command: |
| 79 |
|
| 80 |
strings libcrypto.a | grep -in cert | less |
| 81 |
|
| 82 |
Look in this output for SSL_CERT_DIR and SSL_CERT_FILE, and the lines just |
| 83 |
above them. This is your default location, respectively, for certificates, |
| 84 |
and the CA cert bundle, cert.pem. You will need to know where libcrypto.a is |
| 85 |
found of course. |
| 86 |
|
| 87 |
Example output: |
| 88 |
|
| 89 |
<snip> |
| 90 |
7490:/etc/ssl/certs |
| 91 |
7491:/etc/ssl/cert.pem |
| 92 |
7492:SSL_CERT_DIR |
| 93 |
7493:SSL_CERT_FILE |
| 94 |
<snip> |
| 95 |
|
| 96 |
Other possible example output: |
| 97 |
|
| 98 |
<snip> |
| 99 |
31555:/usr/local/ssl/certs |
| 100 |
31556:/usr/local/ssl/cert.pem |
| 101 |
31557:SSL_CERT_DIR |
| 102 |
31558:SSL_CERT_FILE |
| 103 |
<snip> |
| 104 |
|
| 105 |
Note that when OpenSSL is installed, the c_rehash utility is installed in a |
| 106 |
bin directory (default /usr/local/ssl/bin). You will need to know where it |
| 107 |
is on your system. The command: |
| 108 |
|
| 109 |
whereis c_rehash |
| 110 |
|
| 111 |
will probably give useful results. |
| 112 |
|
| 113 |
Note also that there is no CA cert bundle distributed with OpenSSL. The |
| 114 |
OpenSSL team specifically decided NOT to do that. Getting a set of trusted |
| 115 |
certificates is left up to the installer. |
| 116 |
|
| 117 |
It is no longer a fairly trivial procedure to pull the bundle of trusted root certs out |
| 118 |
of a recent version of Internet Explorer. Multiple certificates are no longer |
| 119 |
exportable as a DER formatted file; extraction of a single certificate is the only |
| 120 |
export for DER, and DER is what converts to PEM. |
| 121 |
|
| 122 |
Users with access to Apple OS X can export all certificates from Keychain Access System Roots as |
| 123 |
a .pem file. Place this in SSL_CERT_DIR and hash it and you're done. |
| 124 |
|
| 125 |
The MirOS BSD project also provides them. The procedure to convert and install them |
| 126 |
is detailed later in this document, and if you simply need to have commercially provided |
| 127 |
certificates trusted by lynx, you can skip down a few lines to the INSTALLING OR UPDATING |
| 128 |
THE CA BUNDLE section. |
| 129 |
|
| 130 |
Extracted Mozilla cert bundles are available for download from the curl project, |
| 131 |
http://curl.haxx.se/docs/caextract.html along with a script to extract from Mozilla |
| 132 |
source. |
| 133 |
|
| 134 |
|
| 135 |
INSTALLING A SELF-SIGNED CERTIFICATE: |
| 136 |
|
| 137 |
When you would like to trust a self-signed (non-commercial) certificate you will |
| 138 |
need to get hold of the actual file. If it's a cert local to your network you |
| 139 |
can ask the sysadmin to make it available for download as a link on a webpage. |
| 140 |
|
| 141 |
If such file is not human-readable it's probably DER formatted and will need to |
| 142 |
be converted to PEM format to allow openssl to use it. |
| 143 |
|
| 144 |
To convert DER formatted certificates into something openssl can deal with: |
| 145 |
|
| 146 |
Save the cert as site_name.crt in a directory. In that directory, type: |
| 147 |
|
| 148 |
openssl x509 -inform DER -in site_name.crt -outform PEM -out site_name.pem |
| 149 |
|
| 150 |
You can now copy this individual cert into the directory for that, usually |
| 151 |
/usr/local/ssl/certs. The alternative is to concatenate the individual certs |
| 152 |
to the cert.pem bundle in /usr/local/ssl. (Please see INSTALLING OR UPDATING |
| 153 |
THE CA BUNDLE below). |
| 154 |
|
| 155 |
The cert file will now be in an acceptable format to openssl, PEM encoded. |
| 156 |
However, openssl, and by extension lynx, will not know about it until that |
| 157 |
cert is symbolically linked to a file named after the hash value of that cert, |
| 158 |
in the default directory /usr/local/ssl/certs. |
| 159 |
|
| 160 |
So the next thing to do is to hash the cert using c_rehash. |
| 161 |
|
| 162 |
INSTALLING OR UPDATING THE CA BUNDLE: |
| 163 |
|
| 164 |
Now would be a good time to check to see if you have the bundle of CA certs |
| 165 |
/usr/local/ssl/cert.pem, or to update them. |
| 166 |
|
| 167 |
CA bundles are available in various places, such as the MirOS BSD distribution, |
| 168 |
for those who want to take that route, or you can extract the current bundle |
| 169 |
from a current version of Internet Explorer (export them all from IE and |
| 170 |
transfer it onto your system). |
| 171 |
|
| 172 |
From MirOS, a cert bundle is available at |
| 173 |
|
| 174 |
http://caunter.ca/ssl.certs.shar |
| 175 |
|
| 176 |
It includes the cacert.org certificate. Download the latest revision; read the |
| 177 |
file to see how to get the certs out. |
| 178 |
|
| 179 |
No hashing is necessary with this set of certs; it is already done; ignore |
| 180 |
the c_rehash usage below for this bundle. Simply run `sh ssl.certs.shar` |
| 181 |
in SSL_CERT_DIR. |
| 182 |
|
| 183 |
From IE 5.x certs extract as a PKCS7 file and need to be converted with something |
| 184 |
like: |
| 185 |
|
| 186 |
openssl pkcs7 -inform DER -in bundle.crt -outform PEM -out cert.pem \ |
| 187 |
-print_certs -text |
| 188 |
|
| 189 |
The resulting cert.pem file should be copied to the default directory for |
| 190 |
bundles (usually /usr/local/ssl) and renamed to "cert.pem", assuming that is |
| 191 |
the SSL_CERT_FILE. |
| 192 |
|
| 193 |
Individual certs can also process if added and hashed in /usr/local/ssl/certs. |
| 194 |
|
| 195 |
We now have all of the individual certs we wish to trust in our certs |
| 196 |
directory, and the most recent bundle of CA certs as well. |
| 197 |
|
| 198 |
Confirm that you have the script c_rehash (See PRELIMINARY PROCEDURES; if it is |
| 199 |
not found, a copy is usually located in the tools directory of the openssl |
| 200 |
source tree. If you use this copy, it needs the execute bit set or it will not |
| 201 |
run). |
| 202 |
|
| 203 |
Run: |
| 204 |
|
| 205 |
./c_rehash |
| 206 |
|
| 207 |
The c_rehash utility is a perl script that runs openssl commands which creates |
| 208 |
the files named after the hash values of the certs in the default directory |
| 209 |
for certs. |
| 210 |
|
| 211 |
Its output looks like this: |
| 212 |
|
| 213 |
Doing /usr/local/ssl/certs |
| 214 |
vsignss.pem => f73e89fd.0 |
| 215 |
vsign3.pem => 7651b327.0 |
| 216 |
...more output |
| 217 |
<snip> |
| 218 |
|
| 219 |
All pem encoded certs in /usr/local/ssl/certs will now be recognized. |
| 220 |
|
| 221 |
SETTING AND EXPORTING ENVIRONMENT VARIABLES: |
| 222 |
|
| 223 |
If lynx is still not recognizing certs, environment variables need |
| 224 |
to be set; if on a sh type shell, the variables also need to be exported. |
| 225 |
|
| 226 |
The environment variables SSL_CERT_DIR and SSL_CERT_FILE need to be set |
| 227 |
if a non-default location is used for certificates, or if certs just can't be |
| 228 |
found by lynx. They may be set as follows in /etc/profile, or a shell |
| 229 |
initialization .profile or .*shrc, if we run a non csh type shell, according |
| 230 |
to the results of the search for the default location for certs procedure |
| 231 |
(See PRELIMINARY PROCEDURES): |
| 232 |
|
| 233 |
SSL_CERT_DIR="/usr/local/ssl/certs" |
| 234 |
SSL_CERT_FILE="/usr/local/ssl/cert.pem" |
| 235 |
export SSL_CERT_DIR SSL_CERT_FILE |
| 236 |
|
| 237 |
On csh type shells, you can use: |
| 238 |
setenv SSL_CERT_DIR "/usr/local/ssl/certs" |
| 239 |
setenv SSL_CERT_FILE "/usr/local/ssl/cert.pem" |
| 240 |
|
| 241 |
Note that the environment variable SSL_CERT_FILE applies to the cert-bundle |
| 242 |
if used outside of the default location (/usr/local/ssl/cert.pem) compiled |
| 243 |
into OpenSSL. There are issues with SSL_CERT_FILE in 0.9.6x versions of openssl. |
| 244 |
|
| 245 |
The configuration file lynx.cfg allows a system SSL_CERT_FILE variable to be set |
| 246 |
which can simplify matters. |
| 247 |
|
| 248 |
SSL_CERT_FILE:/etc/ssl/certs/ca-certificates.crt |
| 249 |
|
| 250 |
Make sure you have FORCE_SSL_PROMPT set to PROMPT in lynx.cfg like so: |
| 251 |
|
| 252 |
FORCE_SSL_PROMPT:PROMPT |
| 253 |
|
| 254 |
You will now connect without error to https servers with trusted certs, but |
| 255 |
will still get this error for untrusted certs: |
| 256 |
|
| 257 |
SSL error:self signed certificate-Continue? (y) |
| 258 |
|
| 259 |
A quick check confirms that these procedures have the same effect with ssl |
| 260 |
errors in the pine program. |
| 261 |
|
| 262 |
2003 updated 2009 |
| 263 |
Stefan Caunter <[email protected]> |
| 264 |
Mohawk College Department of Computer Science |
| 265 |
Hamilton Ontario Canada |