| 1 |
The state from 1.32.0 on: |
| 2 |
========================= |
| 3 |
|
| 4 |
Yet again, things had to change. The previous state works fine for Unix (adjacent) |
| 5 |
sytems, also MinGW stuff on Windows, but it still suffers from off_t being shifty |
| 6 |
and not behaving as I expect, even, when people try to build mpg123 with rogue |
| 7 |
compilers like MSVC, where off_t is always 32 bits, no largefile support, the whole |
| 8 |
idea falls flat. I was pushed to introduce API without off_t where the user can |
| 9 |
provide I/O and ensure largefile support and whatnot without being encumbered by |
| 10 |
offsets possibly limited to 32 bits. |
| 11 |
|
| 12 |
So this is a new set of API functions with the explicit suffix 64 (not _64), notably |
| 13 |
simple functions like |
| 14 |
|
| 15 |
int64_t mpg123_tell64() |
| 16 |
|
| 17 |
but also |
| 18 |
|
| 19 |
int mpg123_reader64() |
| 20 |
|
| 21 |
which takes callbacks that work on int64_t, always. The internal reader code path |
| 22 |
in libmpg123 uses either 32 bit or 64 bit offsets, but throughout the library, |
| 23 |
it is 64 bits being used as if we supported large files everywhere. And in fact, |
| 24 |
we do, if the user provides callbacks that are smart enough. |
| 25 |
|
| 26 |
All existing offset-sensitive code paths are to be rephrased in terms of the new |
| 27 |
portable API (portable, because the platform specifics of off_t implementation are |
| 28 |
not present). All old API entry points in that regard become wrappers over this |
| 29 |
portable API. |
| 30 |
|
| 31 |
The build of libmpg123 needs explicit awareness of largefile support to properly |
| 32 |
reason about which wrapper uses which types. It makes use of off_t at default |
| 33 |
size and explicit off64_t if available. This results in this for a typical 32 bit |
| 34 |
Linux system: |
| 35 |
|
| 36 |
int64_t mpg123_tell64(); // passing on the internal int64_t value |
| 37 |
off_t mpg123_tell(); // converting to 32 bit off_t (overflow check) |
| 38 |
off_t mpg123_tell_32(); // converting to 32 bit off_t (overflow check) |
| 39 |
off64_t mpg123_tell_64(); // converting to 64 bit off_t (no overflow) |
| 40 |
|
| 41 |
A 64 bit Linux would have only these: |
| 42 |
|
| 43 |
int64_t mpg123_tell64(); // passing on the internal int64_t value |
| 44 |
off_t mpg123_tell(); // converting to 64 bit off_t (no overflow) |
| 45 |
off_t mpg123_tell_64(); // converting to 64 bit off_t (no overflow) |
| 46 |
|
| 47 |
This would look the same on a 32 bit system that also went for fixed 64 bit offsets, |
| 48 |
or if you |
| 49 |
|
| 50 |
#define _FILE_OFFSET_BITS 64 |
| 51 |
|
| 52 |
during build on a sensitive system. On a funny system that has 64 bit native integers |
| 53 |
but still 32 bit off_t, this would also be: |
| 54 |
|
| 55 |
int64_t mpg123_tell64(); // passing on the internal int64_t value |
| 56 |
off_t mpg123_tell(); // converting to 32 bit off_t (overflow check) |
| 57 |
off_t mpg123_tell_32(); // converting to 32 bit off_t (overflow check) |
| 58 |
|
| 59 |
If this weird system would be also largefile-sensitive, you get the additional |
| 60 |
|
| 61 |
off64_t mpg123_tell_64(); // converting to 64 bit off_t (no overflow) |
| 62 |
|
| 63 |
entry point. |
| 64 |
|
| 65 |
The whole point here: If you stick to mpg123_tell64(), you always can assume |
| 66 |
64 bit offsets on any platform. If you also provide your own reader functions, you |
| 67 |
can ensure it. Even if we don't properly do large file I/O in libmpg123 itself in |
| 68 |
an MSVC build, users can provide it with their application. Providing a full mpg123 |
| 69 |
program binary along with it in that environment is not a current target. You want |
| 70 |
the library. |
| 71 |
|
| 72 |
The point that every platform gets the _64 suffix means that the header can still |
| 73 |
do the renaming of function calls if client code defines _FILE_OFFSET_BITS and not |
| 74 |
depend on any build-time switches for that. This is what we do officially now with |
| 75 |
headers that are not preprocessed anymore, independent of platform. |
| 76 |
|
| 77 |
The build, just assuming int64 is available, needs to know some bits: |
| 78 |
|
| 79 |
- SIZEOF_OFF_T==8: Is the system 64 bit clean, anyway, with 64 bit off_t? |
| 80 |
- LFS_LARGEFILE_64: Is there explicit 64 bit I/O for the 32 bit system? |
| 81 |
This includes availability of off64_t. |
| 82 |
|
| 83 |
If SIZEOF_OFF_T==8, there will be all 64 bit internal code and two wrappers: |
| 84 |
|
| 85 |
off_t mpg123_tell(); // converting to 64 bit off_t (no overflow) |
| 86 |
off_t mpg123_tell_64(); // converting to 64 bit off_t (no overflow) |
| 87 |
|
| 88 |
The second question isn't even needed. If SIZEOF_OFF_T==4, you get |
| 89 |
|
| 90 |
off_t mpg123_tell(); // converting to 32 bit off_t (overflow check) |
| 91 |
off_t mpg123_tell_32(); // converting to 32 bit off_t (overflow check) |
| 92 |
|
| 93 |
and the additional decision of LFS_LARGEFILE_64 gives actual 64 bit internal |
| 94 |
reader code and the wrapper |
| 95 |
|
| 96 |
off64_t mpg123_tell_64(); // converting to 64 bit off_t (no overflow) |
| 97 |
|
| 98 |
along with that. There is nothing more to know about a system. I do not care how |
| 99 |
large your long is. I only ask |
| 100 |
|
| 101 |
1. How big is your off_t? |
| 102 |
2. If small, can you give me some off64_t? |
| 103 |
|
| 104 |
That's all what there is to it. The build is not even interested in whether off_t |
| 105 |
changes size. This is a detail on the client side that it then gets the off64_t |
| 106 |
versions. I do not support a system that allows changing off_t size _without_ |
| 107 |
off64_t being available. |
| 108 |
|
| 109 |
|
| 110 |
The state from 1.15.4 on: |
| 111 |
========================= |
| 112 |
|
| 113 |
Regarding largefile setup, client apps can be built three ways: |
| 114 |
|
| 115 |
1. _FILE_OFFSET_BITS == 64 (header maps to mpg123_open_64) |
| 116 |
2. _FILE_OFFSET_BITS == 32 (header maps to mpg123_open_32) |
| 117 |
3. _FILE_OFFSET_BITS == <nothing> (header maps to mpg123_open) |
| 118 |
|
| 119 |
The libmpg123 build needs to be prepared for everything. Also, it needs to keep |
| 120 |
in mind the days before introducing large file support --- binaries should still |
| 121 |
work with updated libmpg123. So, mpg123_open should always match what is the |
| 122 |
default build on a platform without added settings. Those are the platform |
| 123 |
variants: |
| 124 |
|
| 125 |
1. 64 bit native system, long == off_t |
| 126 |
libmpg123: mpg123_open |
| 127 |
lfs_alias: mpg123_open_64 -> mpg123_open |
| 128 |
lfs_wrap: <none> |
| 129 |
|
| 130 |
2. largefile-sensitive, long = 32, off_t = 64 (if enabled) |
| 131 |
libmpg123: mpg123_open_64 |
| 132 |
lfs_alias: mpg123_open_32 -> mpg123_open |
| 133 |
lfs_wrap: mpg123_open -> mpg123_open_64 |
| 134 |
|
| 135 |
3. largefile, long = 32, off_t = 64 (FreeBSD) |
| 136 |
libmpg123: mpg123_open |
| 137 |
lfs_alias: mpg123_open_64 -> mpg123_open |
| 138 |
lfs_wrap: <none> |
| 139 |
|
| 140 |
This is what mpg123 does in version 1.15.4 and it works. Well, for cases 1 |
| 141 |
(Linux/Solaris x86-64) and 2 (Linux/Solaris x86). Case 3 needs to be added |
| 142 |
properly. Actually, let's have a second look at case 2: When mpg123 is built |
| 143 |
with --disable-largefile: |
| 144 |
|
| 145 |
2a. largefile-sensitive, mpg123 built with off_t = 32 == long |
| 146 |
libmpg123: mpg123_open |
| 147 |
lfs_alias: mpg123_open_32 -> mpg123_open |
| 148 |
lfs_wrap: <none> |
| 149 |
|
| 150 |
So, this is still correct. Now, what about case 3? What does mpg123 do |
| 151 |
currently, as of 1.15.4? |
| 152 |
|
| 153 |
3a. largefile, long = 32, off_t = 64 (... and mpg123 not really aware of that) |
| 154 |
libmpg123: mpg123_open |
| 155 |
lfs_alias: mpg123_open_32(long) -> mpg123_open(off_t) |
| 156 |
lfs_wrap: <none> |
| 157 |
|
| 158 |
This is _wrong_. Luckily, this does not cause binary compatibility issues, as |
| 159 |
mpg123_open_32 won't be called by anyone unless that someone tries to define |
| 160 |
_FILE_OFFSET_BITS=32, which is nonsense. Perhaps old FreeBSD binaries before |
| 161 |
LFS times? Well, back then, there was no libmpg123. So let's ignore that case. |
| 162 |
The issue at hand is that the alias should be from mpg123_open_64 to |
| 163 |
mpg123_open, for clients that insist on defining _FILE_OFFSET_BITS=64. |
| 164 |
|
| 165 |
The change needed now is to fix the naming and also change the type the |
| 166 |
alias functions use: It is not long int anymore! |
| 167 |
|
| 168 |
Let's revisit case 1 for a moment: My old lfs_alias.c provides for the case |
| 169 |
lfs_alias: mpg123_open -> mpg123_open_64. Is that actually possible? |
| 170 |
What means enforcing _FILE_OFFSET_BITS=64 from the outside, which _could_ |
| 171 |
happen when libmpg123 is included someplace and folks are on the wrong side |
| 172 |
of paranoid regarding this. So, there is |
| 173 |
|
| 174 |
1a. 64 bit native system, long == off_t = 64 and _FILE_OFFSET_BITS=64 |
| 175 |
libmpg123: mpg123_open_64 |
| 176 |
lfs_alias: mpg123_open -> mpg123_open_64 |
| 177 |
lfs_wrap: <none> |
| 178 |
|
| 179 |
(Works also for any system with long == off_t in any width) |
| 180 |
Likewise, there is largefile-sensitive system with enforced 32 bits: |
| 181 |
|
| 182 |
2b. largefile-sensitive, mpg123 with enforced _FILE_OFFSET_BITS=32 |
| 183 |
libmpg123: mpg123_open_32 |
| 184 |
lfs_alias: mpg123_open -> mpg123_open_32 |
| 185 |
lfs_wrap: <none> |
| 186 |
|
| 187 |
All cases are supported with this significant change from 1.15.4: |
| 188 |
Make the aliases use a defined lfs_alias_t, which can be long or off_t, |
| 189 |
depending on what is the default type for offsets on the platform. |
| 190 |
Folks who try _FILE_OFFSET_BITS=32 on a system that only supports |
| 191 |
64 bit get a linking error during mpg123 build (from the _64 aliases), |
| 192 |
which I consider to be a feature. |
| 193 |
|
| 194 |
I salute anyone who is not confused after reading this. |