| 1 |
package Rinci::Transaction; # just to make PodWeaver happy |
| 2 |
|
| 3 |
# AUTHORITY |
| 4 |
# DATE |
| 5 |
our $DIST = 'Rinci'; # DIST |
| 6 |
# VERSION |
| 7 |
|
| 8 |
1; |
| 9 |
# ABSTRACT: A transactional system based on functions |
| 10 |
|
| 11 |
__END__ |
| 12 |
|
| 13 |
=pod |
| 14 |
|
| 15 |
=encoding UTF-8 |
| 16 |
|
| 17 |
=head1 NAME |
| 18 |
|
| 19 |
Rinci::Transaction - A transactional system based on functions |
| 20 |
|
| 21 |
=head1 SPECIFICATION VERSION |
| 22 |
|
| 23 |
Rinci 1.1, protocol version 2 |
| 24 |
|
| 25 |
=head1 VERSION |
| 26 |
|
| 27 |
This document describes version 1.1.104 of Rinci::Transaction (from Perl distribution Rinci), released on 2023-09-30. |
| 28 |
|
| 29 |
=head1 SPECIFICATION |
| 30 |
|
| 31 |
This document describes a transactional system based on functions, where several |
| 32 |
function calls participate in a single transaction. This transactional system |
| 33 |
has the following properties: |
| 34 |
|
| 35 |
=over 4 |
| 36 |
|
| 37 |
=item * Client/server architecture |
| 38 |
|
| 39 |
Transaction can be performed over L<Riap>. Client can start more than one active |
| 40 |
transaction on the server. Each transaction-management request and the function |
| 41 |
calls are requested separately (each one is a separate Riap request). |
| 42 |
|
| 43 |
For more details on this, see L<Riap::Transaction>. |
| 44 |
|
| 45 |
=item * Undo/redo |
| 46 |
|
| 47 |
Committed transactions are still recorded in the database along with its undo |
| 48 |
information. Client can request to undo/redo the transactions. Thus the system |
| 49 |
is also an undo/redo system. |
| 50 |
|
| 51 |
=item * Relies on the functions for reliability/ACID properties |
| 52 |
|
| 53 |
Server or framework provides the transaction manager (TM), but each function |
| 54 |
acts as the resource manager (RM). It is the responsibility of the functions to |
| 55 |
maintain ACID properties while modifying resources. For best results, each |
| 56 |
function should be written carefully and tested extensively, and utilize a real, |
| 57 |
robust RM (like an RDBMS to store data or a transactional filesystem layer to |
| 58 |
read/modify files). In the absence of a real RM, some ACID properties like |
| 59 |
isolation and consistency might be compromised. For example: one transaction TX1 |
| 60 |
modifies a file in an ordinary (i.e. non-transactional) filesystem. Another |
| 61 |
transaction TX2 can see TX1's modification in the middle of uncommitted |
| 62 |
transaction (violates isolation principle). |
| 63 |
|
| 64 |
=back |
| 65 |
|
| 66 |
=head2 How transaction works |
| 67 |
|
| 68 |
The basic idea is that actions are performed by function calls. For each action, |
| 69 |
TM will call the function twice. First to get undo information, and second to |
| 70 |
actually perform the action. The undo information can be used to perform |
| 71 |
rollback, undo, and redo. All functions performing actions in the transaction |
| 72 |
must be able to supply undo information. |
| 73 |
|
| 74 |
=head2 Function requirements |
| 75 |
|
| 76 |
Functions that participate in transaction must declare their C<tx> feature in |
| 77 |
the metadata. In addition, function must also be idempotent. |
| 78 |
|
| 79 |
features => { |
| 80 |
... |
| 81 |
tx => {v=>2}, |
| 82 |
idempotent => 1, |
| 83 |
} |
| 84 |
|
| 85 |
Function must then follow the transaction protocol, described below. |
| 86 |
|
| 87 |
=head2 Transaction manager |
| 88 |
|
| 89 |
The transaction manager manages transaction data and performs actions as well as |
| 90 |
transaction management. |
| 91 |
|
| 92 |
For the sake of examples, our TM stores data in a SQL database (like SQLite) |
| 93 |
with the following tables: |
| 94 |
|
| 95 |
=over 4 |
| 96 |
|
| 97 |
=item * tx |
| 98 |
|
| 99 |
id (PK) |
| 100 |
summary |
| 101 |
ctime (creation time) |
| 102 |
commit_time |
| 103 |
status |
| 104 |
last_action_id -- in-progress action ID (for tx with status=i), or last |
| 105 |
-- processed action (for tx with other transient statuses) |
| 106 |
|
| 107 |
=item * do_action |
| 108 |
|
| 109 |
id (PK) |
| 110 |
tx_id (refers to tx(id)) |
| 111 |
ctime |
| 112 |
sp (savepoint name, UNIQUE(sp,tx_id)) |
| 113 |
f (function name) |
| 114 |
args (arguments, serialized) |
| 115 |
|
| 116 |
=item * undo_action |
| 117 |
|
| 118 |
id (PK) |
| 119 |
tx_id (refers to tx(id)) |
| 120 |
ctime |
| 121 |
f (function name) |
| 122 |
args (arguments, serialized) |
| 123 |
|
| 124 |
=back |
| 125 |
|
| 126 |
=head2 Transaction status |
| 127 |
|
| 128 |
A transaction can have one of these statuses. They will be fully explained in |
| 129 |
the following sections. Statuses having lowercase labels are transient statuses. |
| 130 |
Statuses having uppercase labels are final statuses. |
| 131 |
|
| 132 |
i (in-progress) |
| 133 |
a (aborted, pending rollback to R) |
| 134 |
R (rolled back) |
| 135 |
C (committed) |
| 136 |
u (committed, undoing) |
| 137 |
v (aborted undoing, pending rollback back to C) |
| 138 |
U (committed, undone) |
| 139 |
d (committed & undone, redoing) |
| 140 |
e (aborted redoing, pending rollback back to U) |
| 141 |
X (unresolvable/error) |
| 142 |
|
| 143 |
=head2 Transaction manager initialization |
| 144 |
|
| 145 |
User instantiates TM. TM sets up its data directory and performs cleanup and |
| 146 |
crash recovery. |
| 147 |
|
| 148 |
In cleanup, TM purges unneeded data, like data for rolled back transactions or |
| 149 |
committed transactions that have been around for too long. |
| 150 |
|
| 151 |
In crash recovery, TM looks at all crashed transactions and resolves them |
| 152 |
(either by performing rollback or roll forward). Crashed transactions are |
| 153 |
in-progress (C<i>) transactions that have an in-progress action, or transactions |
| 154 |
having one of these statuses (all the other transient statuses): C<a>, C<u>, |
| 155 |
C<v>, C<d>, C<e>. Crash recovery will be explained in the following sections |
| 156 |
below. |
| 157 |
|
| 158 |
TM also can perform rollback for in-progress transactions that have been around |
| 159 |
for too long without being committed or rolled back. |
| 160 |
|
| 161 |
=head2 Starting transaction |
| 162 |
|
| 163 |
User invokes C<< $tm->begin(tx_id => $tx_id) >>, providing a unique transaction |
| 164 |
ID C<$tx_id> as identifier for the transaction. C<$tx_id> is an arbitrary string |
| 165 |
with a length between 1 and 200 characters. User can also supply C<summary>, a |
| 166 |
textual description for the transaction. It should not be longer than 1024 |
| 167 |
characters. TM will create an entry for the transaction in its journal: |
| 168 |
|
| 169 |
BEGIN; |
| 170 |
INSERT INTO tx (id,summary,ctime,status) VALUES ($tx_id,$summary,$now,'i') |
| 171 |
COMMIT; |
| 172 |
|
| 173 |
As can be seen, initial transaction status is C<i> (in-progress). |
| 174 |
|
| 175 |
Upon success, TM must return status 200. If transaction with that ID already |
| 176 |
exists, TM must return status 409, unless when the existing transaction is still |
| 177 |
on-going, in which case TM should just return 200. TM must return 400 if no |
| 178 |
$tx_id is given. TM can also return status 412 if there are already too many |
| 179 |
transactions being started, either globally on the server or for the particular |
| 180 |
client. |
| 181 |
|
| 182 |
=head2 Performing action |
| 183 |
|
| 184 |
1) User performs action by invoking C<< $tm->action(f=>$fname, args=>$args) >> |
| 185 |
one or several times. Transaction status must be C<i>. TM will first check |
| 186 |
whether function exists and supports transaction. If function does not exist, or |
| 187 |
does not support transaction, TM must return status 412. |
| 188 |
|
| 189 |
2) TM records this action in its journal: |
| 190 |
|
| 191 |
BEGIN; |
| 192 |
INSERT INTO action (tx_id,ctime,f,args) VALUES |
| 193 |
($tx_id,$now,$fname,JSON($args)); -- $act_id |
| 194 |
UPDATE tx SET last_action_id=$act_id WHERE id=$tx_id; |
| 195 |
COMMIT; |
| 196 |
|
| 197 |
3) TM requests state checking and undo information to function, by calling the |
| 198 |
function using the arguments C<$args> and a special argument C<< |
| 199 |
-tx_action=>'check_state' >>. In addition TM also passes C<< -tx_v => N >> (the |
| 200 |
protocol version) and C<< -tx_action_id => UUID >> (a unique identifier to link |
| 201 |
between this call and the 'fix_state' call later). |
| 202 |
|
| 203 |
There are 3 possible states that the function must decide which we are in: |
| 204 |
|
| 205 |
=over 4 |
| 206 |
|
| 207 |
=item * fixed |
| 208 |
|
| 209 |
This is the final, desired state. When we are already in a fixed state, function |
| 210 |
must return status 304 (nothing to do). TM will then skip calling the function |
| 211 |
the second time to fix state, since there is nothing to fix. For example: |
| 212 |
|
| 213 |
[304, "File $path already exists"] # e.g., in a create_file() function |
| 214 |
[304, "User $u already does not exist"] # e.g., in a delete_user() function |
| 215 |
|
| 216 |
=item * fixable |
| 217 |
|
| 218 |
This is where the final, desired state has not been reached, but it is possible |
| 219 |
to reach it. When we are in this state, function must return status 200 with the |
| 220 |
result metadata C<undo_actions>. The message should also describe what needs to |
| 221 |
be fixed. |
| 222 |
|
| 223 |
For example: |
| 224 |
|
| 225 |
[200, "Directory $path needs to be created", undef, |
| 226 |
{undo_actions => [ [rmdir => {path=>$path}] ]}] # e.g. in a mkdir() function |
| 227 |
[200, "User $u should be created with UID $uid", undef, |
| 228 |
{undo_actions => [ [delete_user=>{user=>$u}] ]}] # e.g. in create_user() |
| 229 |
|
| 230 |
=item * unfixable |
| 231 |
|
| 232 |
This is where the final, desired state has not been reached, and it is |
| 233 |
impossible or inappropriate for the function to fix into the fixed state. This |
| 234 |
state is used to avoid undoing what was not fixed by the function. If we are in |
| 235 |
this state, function should return status 412 (precondition failed). |
| 236 |
|
| 237 |
For example: |
| 238 |
|
| 239 |
[412, "Path $path exists but not a symlink"] # e.g. in setup_symlink() |
| 240 |
[412, "User $u exists but with different UID $cur_uid (needs $uid)"] |
| 241 |
|
| 242 |
=back |
| 243 |
|
| 244 |
If state is unfixable, or function returns other statuses (assumed as failure), |
| 245 |
TM stops the process and starts a rollback. C<< $tm->action() >> will return |
| 246 |
with the function's result. |
| 247 |
|
| 248 |
For example, let us use function C<My::setup_unix_user()> which can create a |
| 249 |
Unix user with an empty home directory if the user has not been created. This |
| 250 |
function utilizes several simpler functions: C<My::adduser()> to add entry to |
| 251 |
/etc/passwd and /etc/shadow, C<My::addgroup> to add entry to /etc/group and |
| 252 |
/etc/gshadow, C<My::mkdir> to create directory. Then there are also these |
| 253 |
functions for the undo actions: C<My::deluser> to delete user entry in Unix |
| 254 |
passwd database, C<My::delgroup> to delete group entry in Unix group database, |
| 255 |
and C<My::rmdir> to remove directory. |
| 256 |
|
| 257 |
For C<My::adduser>, the fixable state is that the user does not exist, the fixed |
| 258 |
state is that the user exists. For C<My::deluser>, the fixable state is that |
| 259 |
user exists (additionally with the same UID as the one created previously), the |
| 260 |
fixed state is user does not exist, the unfixable state is user exists but with |
| 261 |
different UID. For C<My::addgroup>, the fixable state is that group does not |
| 262 |
exist, the fixed state is that the group exists. For C<My::delgroup>, the |
| 263 |
fixable state is that group exists (additionally with the same GID as the one |
| 264 |
created previously), the fixed state is group does not exist, the unfixable |
| 265 |
state is group exists but with different GID. For C<My::mkdir>, the fixable |
| 266 |
state is path does not exist, the fixed state is directory exists, and unfixable |
| 267 |
state is path exists but is not a directory. For C<My::rmdir>, the fixable state |
| 268 |
is directory exists and empty, the fixed state is path does not exist, the |
| 269 |
unfixable state is path exists but not a directory or directory is not empty. |
| 270 |
|
| 271 |
The C<undo_actions> must be an array containing action information, in reverse |
| 272 |
order. Each action is a two-element array C<[$fname, $args]> where C<$fname> is |
| 273 |
name of a function (not necessarily the same function) and C<$args> its call |
| 274 |
arguments. |
| 275 |
|
| 276 |
For example, if user invokes C<< $tm->action(f=>'My::setup_unix_user', |
| 277 |
args=>{user=>'bob'}) >> and user C<bob> does not exist yet, function will |
| 278 |
return: |
| 279 |
|
| 280 |
[200, "OK", undef, |
| 281 |
{undo_actions=>[ |
| 282 |
['My::deluser', {group=>'bob'}], |
| 283 |
['My::delgroup', {group=>'bob'}], |
| 284 |
['My::rmdir', {path=>'/home/bob'}], |
| 285 |
]}, |
| 286 |
] |
| 287 |
|
| 288 |
4) TM records these undo actions in its journal: |
| 289 |
|
| 290 |
BEGIN; |
| 291 |
INSERT INTO undo_action (tx_id,ctime,action_id,f,args) VALUES |
| 292 |
($tx_id,$now,$act_id,'My::deluser','{"group":"bob"}'); -- # $uact_id1 |
| 293 |
INSERT INTO undo_action (tx_id,ctime,action_id,f,args) VALUES |
| 294 |
($tx_id,$now,$act_id,'My::delgroup','{"user":"bob"}'); -- # $uact_id2 |
| 295 |
INSERT INTO undo_action (tx_id,ctime,action_id,f,args) VALUES |
| 296 |
($tx_id,$now,$act_id,'My::rmdir','{"path":"/home/bob"}'); -- # $uact_id3 |
| 297 |
COMMIT; |
| 298 |
|
| 299 |
5) If we are in fixed state, this step is skipped. |
| 300 |
|
| 301 |
If we are in fixable state, TM calls function the second time, this time with |
| 302 |
C<< -tx_action => 'fix_state' >>. TM also passes C<-tx_v> and C<-tx_action_id> |
| 303 |
with the same value as the one passed previously during the 'check_state' call. |
| 304 |
Function must perform action to fix the state into the fixed state. In our |
| 305 |
example, C<setup_unix_user()> should create user and group C<bob>, and creates |
| 306 |
an empty directory C</home/bob>. |
| 307 |
|
| 308 |
Function must return status 200 on success. Other status will be interpreted as |
| 309 |
failure, in which case TM will stop the process and starts rollback. C<< |
| 310 |
$tm->action() >> will return with the function's result. |
| 311 |
|
| 312 |
Note: During the 'check_state' phase in step 3, function can also optionally |
| 313 |
return C<do_actions> in its result metadata, for example: |
| 314 |
|
| 315 |
[200, "OK", undef, |
| 316 |
{do_actions=>[ |
| 317 |
['My::adduser', {group=>'bob'}], |
| 318 |
['My::addgroup', {group=>'bob'}], |
| 319 |
['My::mkdir', {path=>'/home/bob'}], |
| 320 |
], |
| 321 |
undo_actions=>[ |
| 322 |
['My::deluser', {group=>'bob'}], |
| 323 |
['My::delgroup', {group=>'bob'}], |
| 324 |
['My::rmdir', {path=>'/home/bob'}], |
| 325 |
]}, |
| 326 |
] |
| 327 |
|
| 328 |
In this case, instead of calling function the second time, TM will just call the |
| 329 |
actions provided by the function, using a nested C<< $tm->action(actions => |
| 330 |
$do_actions) >>. Step 4 will be skipped since each do action will provide its |
| 331 |
own undo actions. |
| 332 |
|
| 333 |
6) If 'fix_state' phase in step 5 succeeds, the action is finished. TM marks |
| 334 |
this: |
| 335 |
|
| 336 |
BEGIN; |
| 337 |
UPDATE tx SET last_action_id=NULL WHERE id=$tx_id; |
| 338 |
COMMIT; |
| 339 |
|
| 340 |
TM is ready to process another action. |
| 341 |
|
| 342 |
=head3 Crash recovery |
| 343 |
|
| 344 |
Recovery rolls back interrupted in-progress transaction. See L</"Rollback of |
| 345 |
in-progress (status i) transaction"> for more details. |
| 346 |
|
| 347 |
If crash happens after step 1, transaction will not be marked as crash since |
| 348 |
C<last_action_id> has not been set and no recovery is necessary. |
| 349 |
|
| 350 |
If crash happens after step 2 until 5, recovery will be performed by rollback. |
| 351 |
Details of rollback is explained in L</"Rollback of in-progress (status i) |
| 352 |
transaction">. |
| 353 |
|
| 354 |
If crash happens after step 6, transaction will not be marked as crash since |
| 355 |
C<last_action_id> is already unset and no recovery is necessary. |
| 356 |
|
| 357 |
=head2 Commit |
| 358 |
|
| 359 |
To commit transaction, user invokes C<< $tm->commit() >>. Transaction status |
| 360 |
must be C<i> or C<a>. If transaction status is C<a>, transaction must be rolled |
| 361 |
back instead. |
| 362 |
|
| 363 |
TM will mark the transaction status as C<C> (committed) and |
| 364 |
delete all entries in the C<do_action> table since they are no longer needed: |
| 365 |
|
| 366 |
BEGIN; |
| 367 |
UPDATE tx SET status='C' WHERE id=$tx_id; |
| 368 |
DELETE FROM do_action WHERE tx_id=$tx_id; |
| 369 |
COMMIT; |
| 370 |
|
| 371 |
TM still stores the C<undo_actions> entries for some time, to allow undo (and |
| 372 |
redo) of transactions. |
| 373 |
|
| 374 |
If transaction status is C<a>, transaction should be rolled back instead of |
| 375 |
committed. |
| 376 |
|
| 377 |
Transaction status progress: |
| 378 |
|
| 379 |
i -> C |
| 380 |
|
| 381 |
=head2 Rollback of in-progress (status i) transaction |
| 382 |
|
| 383 |
If an action fails, or some other error happens, rollback will be performed by |
| 384 |
TM. Rollback can also be started by user using C<< $tm->rollback >>. TM marks |
| 385 |
transaction status to C<a> (aborted). This will prevent other clients trying to |
| 386 |
add new actions to this transaction, since aborted transaction can longer accept |
| 387 |
new actions, it can only be rolled back. |
| 388 |
|
| 389 |
TM will then perform undo for each function, in reverse order, using the undo |
| 390 |
actions previously recorded in C<undo_action> table. The process is similar to |
| 391 |
performing action, except that: |
| 392 |
|
| 393 |
=over 4 |
| 394 |
|
| 395 |
=item * After rollback succeeds, transaction status is changed to C<R> |
| 396 |
|
| 397 |
C<R> means rolled back. These transactions can be discarded by the next cleanup |
| 398 |
process. |
| 399 |
|
| 400 |
=item * Undo actions are not recorded |
| 401 |
|
| 402 |
Since we do not rollback from the rollback process, but continue it. TM still |
| 403 |
calls function twice for each action (check_state + fix_state), but do not |
| 404 |
bother to record the undo actions returned by function in the check_state phase |
| 405 |
to its database. |
| 406 |
|
| 407 |
=item * Failure in rollback step will mark transaction status as C<X> |
| 408 |
|
| 409 |
C<X> means inconsistent/error. Transactions left in this state are probably |
| 410 |
half-done and thus inconsistent. We give up on these transactions and the next |
| 411 |
cleanup process can discard them. |
| 412 |
|
| 413 |
(TODO: Should there be an option to continue to the next action anyway? But this |
| 414 |
is not necessarily more robust or correct.) |
| 415 |
|
| 416 |
=back |
| 417 |
|
| 418 |
Transaction status progress: |
| 419 |
|
| 420 |
i -> a -> R # successful rollback |
| 421 |
i -> a -> X # failed rollback |
| 422 |
|
| 423 |
B<Example>. Continuing our previous example, in the C<< |
| 424 |
setup_unix_user(user=>'bob') >> action, there are 3 actions involved: |
| 425 |
|
| 426 |
['My::adduser', {group=>'bob'}] |
| 427 |
['My::addgroup', {group=>'bob'}] |
| 428 |
['My::mkdir', {path=>'/home/bob'}] |
| 429 |
|
| 430 |
Suppose action 1 and 2 succeed, and the following undo actions have been |
| 431 |
recorded in C<undo_action>: |
| 432 |
|
| 433 |
['My::deluser', {group=>'bob'}] # recorded with ID $ucall_id1 |
| 434 |
['My::delgroup', {group=>'bob'}] # recorded with ID $ucall_id2 |
| 435 |
|
| 436 |
Suppose action 3 fails with status 500 (e.g. permission denied) and thus |
| 437 |
rollback is started. The following is the steps that happen during rollback. |
| 438 |
Actions will be processed in reverse order: C<$ucall_id2>, C<$ucall_id1>. |
| 439 |
|
| 440 |
1) TM marks transaction status to aborted: |
| 441 |
|
| 442 |
BEGIN; |
| 443 |
UPDATE tx SET status='a', last_action_id=NULL WHERE id=$tx_id; |
| 444 |
COMMIT; |
| 445 |
|
| 446 |
TM performs action C<My::delgroup>. |
| 447 |
|
| 448 |
2a) TM calls C<My::delgroup()> the first time with C<< -tx_action => |
| 449 |
'check_state' >>. TM also passes C<< -tx_is_rollback => 1 >> for informative |
| 450 |
purposes (some function can utilize this information to behave more robust, for |
| 451 |
example, to avoid failing the rollback process). TM does not record the |
| 452 |
C<undo_actions> metadata returned, but observes the C<do_actions>. |
| 453 |
|
| 454 |
If function returns 304, step 2b is skipped and TM moves on to the next action. |
| 455 |
If function returns 200, TM continues to step 2b. If function returns other |
| 456 |
statuses, TM assumes rollback failure and marks transaction as C<X> and ends the |
| 457 |
rollback process for this transaction. |
| 458 |
|
| 459 |
2b) TM invokes C<My::delgroup()> the second time to perform the action, passing |
| 460 |
C<< -tx_action => 'fix_state' >> and C<< -tx_is_rollback => 1 >>. Function sees |
| 461 |
that group exists (fixable state), deletes it, return status 200. |
| 462 |
|
| 463 |
2c) TM sets transaction's C<last_action_id> to C<$uact_id1> to mark that this |
| 464 |
action has been processed: |
| 465 |
|
| 466 |
BEGIN; |
| 467 |
UPDATE tx SET last_action_id=$ucall_id1 WHERE id=$tx_id; |
| 468 |
COMMIT; |
| 469 |
|
| 470 |
TM then continues to perform action C<My::delgroup>. |
| 471 |
|
| 472 |
3a) Just like in step 2, TM invokes C<My::deluser()> the first time to check |
| 473 |
state. |
| 474 |
|
| 475 |
3b) TM invokes C<My::deluser()> to perform the action. Function sees that user |
| 476 |
exists (fixable state), deletes it, return status 200. |
| 477 |
|
| 478 |
3c) TM sets transaction's C<last_action_id> to C<$uact_id2> to mark that this |
| 479 |
action has been processed: |
| 480 |
|
| 481 |
BEGIN; |
| 482 |
UPDATE tx SET last_action_id=$uact_id2 WHERE id=$tx_id; |
| 483 |
COMMIT; |
| 484 |
|
| 485 |
4) TM completes the rollback process by setting transaction status to C<R>. |
| 486 |
|
| 487 |
BEGIN; |
| 488 |
UPDATE tx SET status='R' WHERE id=$tx_id; |
| 489 |
COMMIT; |
| 490 |
|
| 491 |
By now the effect of the transaction has been nullified. |
| 492 |
|
| 493 |
=head3 * Crash recovery |
| 494 |
|
| 495 |
Recovery continues the interrupted rollback process. |
| 496 |
|
| 497 |
If crash happens after step 1, recovery will continue the rollback process. |
| 498 |
Rollback of aborted (status a) transaction is exactly the same as rollback of |
| 499 |
in-progress (status i) transaction, except that C<last_action_id> is not reset. |
| 500 |
|
| 501 |
If crash happens after step 2a-2b, C<last_action_id> is still unset, so the |
| 502 |
process resumes at step 2a. TM does not remember whether previously before crash |
| 503 |
the function has been executed (and cannot remember, the progress of the |
| 504 |
execution inside the function). This is the reason why function needs to be |
| 505 |
idempotent, because it is potentially executed twice by TM for the same action. |
| 506 |
If function has completed deleting the group before crash, C<check_state> will |
| 507 |
return status 304 (fixed) and TM will skip step 2b. If function has not deleted |
| 508 |
the group before crash, C<check_state> will return status 200 (fixable) and TM |
| 509 |
will execute step 2b. |
| 510 |
|
| 511 |
If crash happens after step 2c/3a-3b, C<last_action_id> is set to C<$uact_id1>. |
| 512 |
Process will resume at step 3a, since $uact_id1 has been marked as done. |
| 513 |
|
| 514 |
If crash happens after step 3c, process will resume at step 4. |
| 515 |
|
| 516 |
If crash happens after step 4, no recovery is necessary since transaction has |
| 517 |
been rolled back completely. |
| 518 |
|
| 519 |
=head2 Undo |
| 520 |
|
| 521 |
TM allows undoing committed transaction, so the transaction system also serves |
| 522 |
as an undo/redo system. |
| 523 |
|
| 524 |
1) User performs undo by invoking C<< $tm->undo(tx_id => $tx_id) >>, where |
| 525 |
C<$tx_id> is the ID of a committed transaction. If C<$tx_id> is not supplied, |
| 526 |
the client's newest committed transaction is used. TM will first check that |
| 527 |
transaction status is indeed C<C>. |
| 528 |
|
| 529 |
2) TM sets transaction status to C<u> (undoing): |
| 530 |
|
| 531 |
BEGIN; |
| 532 |
UPDATE tx SET status='u' WHERE id=$tx_id; |
| 533 |
COMMIT; |
| 534 |
|
| 535 |
TM then performs actions specified in the C<undo_action> table. The process is |
| 536 |
similar to performing action, except: |
| 537 |
|
| 538 |
=over 4 |
| 539 |
|
| 540 |
=item * After undo succeeds, transaction status is changed to C<U> |
| 541 |
|
| 542 |
C<U> means committed but undone transaction. These transactions can be redone |
| 543 |
back to status C<C>. |
| 544 |
|
| 545 |
=item * Undo actions are recorded in C<do_action> table instead of C<undo_action> |
| 546 |
|
| 547 |
=item * Failure in undo step will cause transaction to roll back to status C<C> |
| 548 |
|
| 549 |
=back |
| 550 |
|
| 551 |
Transaction status progress: |
| 552 |
|
| 553 |
C -> u -> U # successful undo |
| 554 |
C -> u -> v -> C # failed undo, rolled back to C |
| 555 |
|
| 556 |
Continuing our previous example, suppose our C<< setup_unix_user(user=>'bob') >> |
| 557 |
transaction has succeeded and been committed. The C<undo_action> table contains |
| 558 |
these entries: |
| 559 |
|
| 560 |
['My::deluser', {group=>'bob'}] # recorded with ID $uact_id1 |
| 561 |
['My::delgroup', {group=>'bob'}] # recorded with ID $uact_id2 |
| 562 |
['My::rmdir', {path=>'/home/bob'}] # recorded with ID $uact_id3 |
| 563 |
|
| 564 |
Actions will be processed in reverse order: C<$uact_id3>, C<$uact_id2>, |
| 565 |
C<$uact_id1>. |
| 566 |
|
| 567 |
3a) TM invokes C<My::rmdir> the first time with C<< -tx_action => 'check_state' |
| 568 |
>>. If directory has been filled by files/subdirectories, function will return |
| 569 |
412 ("Cannot remove home directory, non-empty") and the undo process fails with |
| 570 |
this status. If directory exists and is still empty, function will return 200 |
| 571 |
(fixable state) and process continues. |
| 572 |
|
| 573 |
3b) TM records the C<undo_actions> result metadata returned by function to |
| 574 |
C<do_action> table, for redo information. |
| 575 |
|
| 576 |
BEGIN; |
| 577 |
INSERT INTO do_action (tx_id,ctime,f,args) VALUES |
| 578 |
($tx_id,$now,'My::mkdir', '{"path":"/home/bob"}'); # -- $ract_id1 |
| 579 |
COMMIT; |
| 580 |
|
| 581 |
3c) TM invokes C<My::rmdir> the second time with C<< -tx_action => 'fix_state' |
| 582 |
>>. Function deletes directory and return 200. |
| 583 |
|
| 584 |
3d) TM updates C<last_action_id> to mark that this action has been processed: |
| 585 |
|
| 586 |
BEGIN; |
| 587 |
UPDATE tx SET last_action_id=$uact_id3 WHERE id=$tx_id; |
| 588 |
COMMIT; |
| 589 |
|
| 590 |
TM then continue to C<$uact_id2>. |
| 591 |
|
| 592 |
4a) TM invokes C<My::delgroup> the first time with C<< -tx_action => |
| 593 |
'check_state' >>. |
| 594 |
|
| 595 |
4b) TM records undo_actions: |
| 596 |
|
| 597 |
BEGIN; |
| 598 |
INSERT INTO do_action (tx_id,ctime,f,args) VALUES |
| 599 |
($tx_id,$now,'My::addgroup', '{"group":"bob"}'); # -- $ract_id2 |
| 600 |
COMMIT; |
| 601 |
|
| 602 |
4c) TM invokes C<My::addgroup> the second time with C<< -tx_action => |
| 603 |
'fix_state' >>. Function sees that group exists, deletes it, and returns 200. |
| 604 |
|
| 605 |
4d) TM updates C<last_action_id>: |
| 606 |
|
| 607 |
BEGIN; |
| 608 |
UPDATE tx SET last_action_id=$uact_id2 WHERE id=$tx_id; |
| 609 |
COMMIT; |
| 610 |
|
| 611 |
TM then continue to C<$uact_id1>. |
| 612 |
|
| 613 |
5a) TM invokes C<My::deluser> the first time with C<< -tx_action => |
| 614 |
'check_state' >>. |
| 615 |
|
| 616 |
5b) TM records undo_actions: |
| 617 |
|
| 618 |
BEGIN; |
| 619 |
INSERT INTO undo_action (tx_id,ctime,f,args) VALUES |
| 620 |
($tx_id,$now,'My::adduser', '{"user":"bob"}'); # -- $ract_id3 |
| 621 |
COMMIT; |
| 622 |
|
| 623 |
5c) TM invokes C<My::adduser> the second time with C<< -tx_action => 'fix_state' |
| 624 |
>>. Function sees that user exists, deletes it, and returns 200. |
| 625 |
|
| 626 |
5d) TM updates C<last_action_id>: |
| 627 |
|
| 628 |
BEGIN; |
| 629 |
UPDATE tx SET last_action_id=$uact_id1 WHERE id=$tx_id; |
| 630 |
COMMIT; |
| 631 |
|
| 632 |
6) TM completes the undo process by setting transaction status to C<U>: |
| 633 |
|
| 634 |
BEGIN; |
| 635 |
UPDATE tx SET status='U', last_action_id=NULL WHERE id=$tx_id; |
| 636 |
COMMIT; |
| 637 |
|
| 638 |
=head3 Crash recovery |
| 639 |
|
| 640 |
Recovery rolls back interrupted undoing process so that transaction status is |
| 641 |
back to C<C> (committed). For more details, refer to L</"Rollback of undoing |
| 642 |
(status u) transaction">. |
| 643 |
|
| 644 |
If crash happens before finishing step 2, no recovery is necessary. |
| 645 |
|
| 646 |
If crash happens after step 2-3c, recovery resumes from step 3a since |
| 647 |
C<last_action_id> is still unset. That is why C<My::mkdir> needs to be |
| 648 |
idempotent and can check state, since it is potentially executed (step 3c) |
| 649 |
twice, before and after recovery. |
| 650 |
|
| 651 |
If crash happens after step 3d-4c, recovery recovery resumes from step 4a since |
| 652 |
C<last_action_id> is set to C<$uact_id3>. |
| 653 |
|
| 654 |
If crash happens after step 4d-5c, recovery resumes from step 5a since |
| 655 |
C<last_action_id> is set to C<$uact_id2>. |
| 656 |
|
| 657 |
If crash happens after step 5d, recovery resumes from step 6. |
| 658 |
|
| 659 |
=head2 Rolling back the undoing (status u) transaction |
| 660 |
|
| 661 |
If undo fails in the middle, rollback will happen. TM marks transaction status |
| 662 |
from C<u> to C<v>, this differentiates between an undo process in progress (in |
| 663 |
which case recovery should continue it until status is C<U>) and a failed undo |
| 664 |
process (in which case recovery should rolls it back to status C<C>). |
| 665 |
|
| 666 |
TM will then perform actions from the C<do_action> table. The process is similar |
| 667 |
to rollback of in-progress (status i) transaction, except that after rollback |
| 668 |
succeeds, transaction status is set to C<C>. |
| 669 |
|
| 670 |
If rollback fails, transaction status is set to C<X>. |
| 671 |
|
| 672 |
Transaction status progress: |
| 673 |
|
| 674 |
u -> v -> C # rollback succeeds |
| 675 |
u -> v -> X # rollback fails |
| 676 |
|
| 677 |
=head3 Crash recovery |
| 678 |
|
| 679 |
Recovery continues the rollback process. |
| 680 |
|
| 681 |
=head2 Redo |
| 682 |
|
| 683 |
An undone transaction (status C<U>) can be redone back to C<C>. To do this, user |
| 684 |
invokes C<< $tm->undo(tx_id => $tx_id) >>, where C<$tx_id> is the ID of an |
| 685 |
undone transaction. If C<$tx_id> is not supplied, the client's newest undone |
| 686 |
transaction is used. TM will first check that transaction status is indeed C<U>. |
| 687 |
|
| 688 |
TM will then set transaction status to C<d> (redoing): |
| 689 |
|
| 690 |
BEGIN; |
| 691 |
UPDATE tx SET status='d' WHERE id=$tx_id; |
| 692 |
COMMIT; |
| 693 |
|
| 694 |
This will prevent other clients trying to redo the same transaction. TM will |
| 695 |
then process actions found in C<do_action> table, just like when performing |
| 696 |
normal action. |
| 697 |
|
| 698 |
Transaction status progress: |
| 699 |
|
| 700 |
U -> d -> C |
| 701 |
|
| 702 |
=head3 Crash recovery |
| 703 |
|
| 704 |
Recovery rolls back the redoing process. See L</"Rolling back a redoing (status |
| 705 |
d) transaction">. |
| 706 |
|
| 707 |
=head2 Rolling back a redoing (status d) transaction |
| 708 |
|
| 709 |
If redo fails in the middle, rollback will happen. TM marks transaction status |
| 710 |
from C<d> to C<e> (failed redo). This will differentiate between a redo process |
| 711 |
in progress (in which case recovery should continue it until status is C<C>) and |
| 712 |
a failed redo process (in which case recovery should rolls it back to status |
| 713 |
C<U>). |
| 714 |
|
| 715 |
TM will perform actions from the C<undo_action> table. The process is similar to |
| 716 |
rollback of an in-progress (status i) transaction, except that after rollback |
| 717 |
succeeds, transaction status is set to C<U>. |
| 718 |
|
| 719 |
If rollback fails, TM will set transaction status to C<X>. |
| 720 |
|
| 721 |
Transaction status progress: |
| 722 |
|
| 723 |
d -> e -> U # rollback succeeds |
| 724 |
d -> e -> X # rollback fails |
| 725 |
|
| 726 |
=head3 Crash recovery |
| 727 |
|
| 728 |
Recovery continues the rollback process. |
| 729 |
|
| 730 |
=head2 Cleanup |
| 731 |
|
| 732 |
Cleanup is done at TM startup and at regular intervals. TM should delete |
| 733 |
(forget) all C and U transactions that are too old, or keep the number of those |
| 734 |
transactions under a certain limit, according to its settings. As soon as those |
| 735 |
transactions are deleted, they can no longer be undone/redone, since the undo |
| 736 |
actions data has been deleted too. |
| 737 |
|
| 738 |
The cleanup process also deletes all X transactions, since they cannot be |
| 739 |
resolved anyway (TODO: perhaps some retry mechanism can be applied, if desired?) |
| 740 |
|
| 741 |
Cleanup process also deletes all R transactions. |
| 742 |
|
| 743 |
Cleanup process can also roll back any transactions with status C<i> that have |
| 744 |
been going for too long without being committed/rolled back. |
| 745 |
|
| 746 |
=head2 Savepoint |
| 747 |
|
| 748 |
Basically savepoint is just a label in the C<do_action> table. |
| 749 |
|
| 750 |
To mark a savepoint, user invokes C<< $tm->savepoint(sp_id=>$sp_id) >> where |
| 751 |
C<$sp_id> is an arbitrary string from 1-64 characters. It must be unique within |
| 752 |
the transaction. If the same savepoint is used, the old savepoint is replaced by |
| 753 |
the new one. |
| 754 |
|
| 755 |
To release (forget) a savepoint, user invokes C<< |
| 756 |
$tm->release_savepoint(sp_id=>$sp_id) >>. It just clears the label in the |
| 757 |
C<do_action> table. |
| 758 |
|
| 759 |
Rollback to a savepoint is just a normal rollback process, except we stop after |
| 760 |
finishing the undo actions of the corresponding action with the savepoint, and |
| 761 |
transaction status is set back to C<i>. If savepoint is unknown (or marked |
| 762 |
before any action, which is effectively the same), we rollback everything in the |
| 763 |
transaction. |
| 764 |
|
| 765 |
=head2 Discard |
| 766 |
|
| 767 |
User can optionally do a cleanup of her transactions by issuing C<< |
| 768 |
$tm->discard(tx_id=>$tx_id) >> or C<< $tm->discard_all >>. Transactions that can |
| 769 |
be discarded are those with the final statuses: C<C>, C<U>, C<X>. |
| 770 |
|
| 771 |
=head1 FAQ |
| 772 |
|
| 773 |
=head2 Why is this useful? |
| 774 |
|
| 775 |
The protocol is a pretty generic and simple way to build transactional system, |
| 776 |
even on heterogenous, multiuser environment. If the functions are written |
| 777 |
carefully, the system can be reliable. And even if some of the ACID properties |
| 778 |
are compromised due to lack of real RM, the system is still useful for its |
| 779 |
undo/redo capability. |
| 780 |
|
| 781 |
=head2 What are the drawbacks? |
| 782 |
|
| 783 |
The reliability of the system rests on the reliability of each involved |
| 784 |
function. One buggy function can break the transaction. |
| 785 |
|
| 786 |
=head2 What about non-undoable actions? |
| 787 |
|
| 788 |
Non-undoable actions (like sending an email, permanently deleting files) should |
| 789 |
be executed outside the scope of transaction. |
| 790 |
|
| 791 |
=head1 HOMEPAGE |
| 792 |
|
| 793 |
Please visit the project's homepage at L<https://metacpan.org/release/Rinci>. |
| 794 |
|
| 795 |
=head1 SOURCE |
| 796 |
|
| 797 |
Source repository is at L<https://github.com/perlancar/perl-Rinci>. |
| 798 |
|
| 799 |
=head1 SEE ALSO |
| 800 |
|
| 801 |
Transaction behavior is largely based on PostgreSQL. |
| 802 |
|
| 803 |
Related specifications: L<Rinci::function>, L<Riap::Transaction> |
| 804 |
|
| 805 |
Implementations: L<Perinci::Tx::Manager> |
| 806 |
|
| 807 |
=head1 AUTHOR |
| 808 |
|
| 809 |
perlancar <[email protected]> |
| 810 |
|
| 811 |
=head1 CONTRIBUTING |
| 812 |
|
| 813 |
|
| 814 |
To contribute, you can send patches by email/via RT, or send pull requests on |
| 815 |
GitHub. |
| 816 |
|
| 817 |
Most of the time, you don't need to build the distribution yourself. You can |
| 818 |
simply modify the code, then test via: |
| 819 |
|
| 820 |
% prove -l |
| 821 |
|
| 822 |
If you want to build the distribution (e.g. to try to install it locally on your |
| 823 |
system), you can install L<Dist::Zilla>, |
| 824 |
L<Dist::Zilla::PluginBundle::Author::PERLANCAR>, |
| 825 |
L<Pod::Weaver::PluginBundle::Author::PERLANCAR>, and sometimes one or two other |
| 826 |
Dist::Zilla- and/or Pod::Weaver plugins. Any additional steps required beyond |
| 827 |
that are considered a bug and can be reported to me. |
| 828 |
|
| 829 |
=head1 COPYRIGHT AND LICENSE |
| 830 |
|
| 831 |
This software is copyright (c) 2023, 2022, 2021, 2020, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012 by perlancar <[email protected]>. |
| 832 |
|
| 833 |
This is free software; you can redistribute it and/or modify it under |
| 834 |
the same terms as the Perl 5 programming language system itself. |
| 835 |
|
| 836 |
=head1 BUGS |
| 837 |
|
| 838 |
Please report any bugs or feature requests on the bugtracker website L<https://rt.cpan.org/Public/Dist/Display.html?Name=Rinci> |
| 839 |
|
| 840 |
When submitting a bug or request, please include a test-file or a |
| 841 |
patch to an existing test-file that illustrates the bug or desired |
| 842 |
feature. |
| 843 |
|
| 844 |
=cut |