Debian Codegraph @

README

raw
1 NAME
2 File::Write::Rotate - Write to files that archive/rotate themselves
3
4 VERSION
5 This document describes version 0.321 of File::Write::Rotate (from Perl
6 distribution File-Write-Rotate), released on 2019-06-27.
7
8 SYNOPSIS
9 use File::Write::Rotate;
10
11 my $fwr = File::Write::Rotate->new(
12 dir => '/var/log', # required
13 prefix => 'myapp', # required
14 #suffix => '.log', # default is ''
15 size => 25*1024*1024, # default is 10MB, unless period is set
16 histories => 12, # default is 10
17 #buffer_size => 100, # default is none
18 );
19
20 # write, will write to /var/log/myapp.log, automatically rotate old log files
21 # to myapp.log.1 when myapp.log reaches 25MB. will keep old log files up to
22 # myapp.log.12.
23 $fwr->write("This is a line\n");
24 $fwr->write("This is", " another line\n");
25
26 To compressing old log files:
27
28 $fwr->compress;
29
30 This is usually done in a separate process, because it potentially takes
31 a long time if the files to compress are large; we are rotating
32 automatically in write() so doing automatic compression too would
33 annoyingly block writer for a potentially long time.
34
35 DESCRIPTION
36 This module can be used to write to file, usually for logging, that can
37 rotate itself. File will be opened in append mode. By default, locking
38 will be done to avoid conflict when there are multiple writers. Rotation
39 can be done by size (after a certain size is reached), by time
40 (daily/monthly/yearly), or both.
41
42 I first wrote this module for logging script STDERR output to files (see
43 Tie::Handle::FileWriteRotate).
44
45 ATTRIBUTES
46 buffer_size => int
47 Get or set buffer size. If set to a value larger than 0, then when a
48 write() failed, instead of dying, the message will be stored in an
49 internal buffer first (a regular Perl array). When the number of items
50 in the buffer exceeds this size, then write() will die upon failure.
51 Otherwise, every write() will try to flush the buffer.
52
53 Can be used for example when a program runs as superuser/root then
54 temporarily drops privilege to a normal user. During this period,
55 logging can fail because the program cannot lock the lock file or write
56 to the logging directory. Before dropping privilege, the program can set
57 buffer_size to some larger-than-zero value to hold the messages emitted
58 during dropping privilege. The next write() as the superuser/root will
59 succeed and flush the buffer to disk (provided there is no other error
60 condition, of course).
61
62 path => str (ro)
63 Current file's path.
64
65 handle => (ro)
66 Current file handle. You should not use this directly, but use write()
67 instead. This attribute is provided for special circumstances (e.g. in
68 hooks, see example in the hook section).
69
70 hook_before_write => code
71 Will be called by write() before actually writing to filehandle (but
72 after locking is done). Code will be passed ($self, \@msgs, $fh) where
73 @msgs is an array of strings to be written (the contents of buffer, if
74 any, plus arguments passed to write()) and $fh is the filehandle.
75
76 hook_before_rotate => code
77 Will be called by the rotating routine before actually doing rotating.
78 Code will be passed ($self).
79
80 This can be used to write a footer to the end of each file, e.g.:
81
82 # hook_before_rotate
83 my ($self) = @_;
84 my $fh = $self->handle;
85 print $fh "Some footer\n";
86
87 Since this hook is indirectly called by write(), locking is already
88 done.
89
90 hook_after_rotate => code
91 Will be called by the rotating routine after the rotating process. Code
92 will be passed ($self, \@renamed, \@deleted) where @renamed is array of
93 new filenames that have been renamed, @deleted is array of new filenames
94 that have been deleted.
95
96 hook_after_create => code
97 Will be called by after a new file is created. Code will be passed
98 ($self).
99
100 This hook can be used to write a header to each file, e.g.:
101
102 # hook_after_create
103 my ($self) = @_;
104 my $fh $self->handle;
105 print $fh "header\n";
106
107 Since this is called indirectly by write(), locking is also already
108 done.
109
110 binmode => str
111 If set to "1", will cause the file handle to be set:
112
113 binmode $fh;
114
115 which might be necessary on some OS, e.g. Windows when writing binary
116 data. Otherwise, other defined values will cause the file handle to be
117 set:
118
119 binmode $fh, $value
120
121 which can be used to set PerlIO layer(s).
122
123 METHODS
124 $obj = File::Write::Rotate->new(%args)
125 Create new object. Known arguments:
126
127 * dir => STR (required)
128
129 Directory to put the files in.
130
131 * prefix => STR (required)
132
133 Name of files. The files will be named like the following:
134
135 <prefix><period><suffix><rotate_suffix>
136
137 "<period>" will only be given if the "period" argument is set. If
138 "period" is set to "yearly", "<period>" will be "YYYY" (4-digit
139 year). If "period" is "monthly", "<period>" will be "YYYY-MM"
140 (4-digit year and 2-digit month). If "period" is "daily", "<period>"
141 will be "YYYY-MM-DD" (4-digit year, 2-digit month, and 2-digit day).
142
143 "<rotate_suffix>" is either empty string for current file; or .1, .2
144 and so on for rotated files. .1 is the most recent rotated file, .2
145 is the next most recent, and so on.
146
147 An example, with "prefix" set to "myapp":
148
149 myapp # current file
150 myapp.1 # most recently rotated
151 myapp.2 # the next most recently rotated
152
153 With "prefix" set to "myapp", "period" set to "monthly", "suffix"
154 set to ".log":
155
156 myapp.2012-12.log # file name for december 2012
157 myapp.2013-01.log # file name for january 2013
158
159 Like previous, but additionally with "size" also set (which will
160 also rotate each period file if it exceeds specified size):
161
162 myapp.2012-12.log # file(s) for december 2012
163 myapp.2012-12.log.1
164 myapp.2012-12.log.2
165 myapp.2013-01.log # file(s) for january 2013
166
167 All times will use local time, so you probably want to set "TZ"
168 environment variable or equivalent methods to set time zone.
169
170 * suffix => STR (default: '')
171
172 Suffix to give to file names, usually file extension like ".log".
173 See "prefix" for more details.
174
175 If you use a yearly period, setting suffix is advised to avoid
176 ambiguity with rotate suffix (for example, is "myapp.2012" the
177 current file for year 2012 or file with 2012 rotate suffix?)
178
179 * size => INT (default: 10*1024*1024)
180
181 Maximum file size, in bytes, before rotation is triggered. The
182 default is 10MB (10*1024*1024) *if* "period" is not set. If "period"
183 is set, no default for "size" is provided, which means files will
184 not be rotated for size (only for period).
185
186 * period => STR
187
188 Can be set to either "daily", "monthly", or "yearly". If set, will
189 automatically rotate after period change. See "prefix" for more
190 details.
191
192 * histories => INT (default: 10)
193
194 Number of rotated files to keep. After the number of files exceeds
195 this, the oldest one will be deleted. 0 means not to keep any
196 history, 1 means to only keep .1 file, and so on.
197
198 * buffer_size => INT (default: 0)
199
200 Set initial value of buffer. See the "buffer_size" attribute for
201 more information.
202
203 * lock_mode => STR (default: 'write')
204
205 Can be set to either "none", "write", or "exclusive". "none"
206 disables locking and increases write performance, but should only be
207 used when there is only one writer. "write" acquires and holds the
208 lock for each write. "exclusive" acquires the lock at object
209 creation and holds it until the the object is destroyed.
210
211 Lock file is named "<prefix>"".lck". Will wait for up to 1 minute to
212 acquire lock, will die if failed to acquire lock.
213
214 * hook_before_write => CODE
215
216 * hook_before_rotate => CODE
217
218 * hook_after_rotate => CODE
219
220 * hook_after_create => CODE
221
222 See "ATTRIBUTES".
223
224 * buffer_size => int
225
226 * rotate_probability => float (between 0 < x < 1)
227
228 If set, instruct to only check for rotation under a certain
229 probability, for example if value is set to 0.1 then will only check
230 for rotation 10% of the time.
231
232 lock_file_path => STR
233 Returns a string representing the complete pathname to the lock file,
234 based on "dir" and "prefix" attributes.
235
236 $fwr->write(@args)
237 Write to file. Will automatically rotate file if period changes or file
238 size exceeds specified limit. When rotating, will only keep a specified
239 number of histories and delete the older ones.
240
241 Does not append newline so you'll have to do it yourself.
242
243 $fwr->compress
244 Compress old rotated files and remove the uncompressed originals.
245 Currently uses IO::Compress::Gzip to do the compression. Extension given
246 to compressed file is ".gz".
247
248 Will not lock writers, but will create "<prefix>""-compress.pid" PID
249 file to prevent multiple compression processes running and to signal the
250 writers to postpone rotation.
251
252 After compression is finished, will remove the PID file, so rotation can
253 be done again on the next "write()" if necessary.
254
255 FAQ
256 Why use autorotating file?
257 Mainly convenience and low maintenance. You no longer need a separate
258 rotator process like the Unix logrotate utility (which when accidentally
259 disabled or misconfigured will cause your logs to stop being rotated and
260 grow indefinitely).
261
262 What is the downside of using FWR (and LDFR)?
263 Mainly (significant) performance overhead. At (almost) every "write()",
264 FWR needs to check file sizes and/or dates for rotation. Under default
265 configuration (where "lock_mode" is "write"), it also performs locking
266 on each "write()" to make it safe to use with multiple processes. Below
267 is a casual benchmark to give a sense of the overhead, tested on my Core
268 i5-2400 3.1GHz desktop:
269
270 Writing lines in the size of ~ 200 bytes, raw writing to disk (SSD) has
271 the speed of around 3.4mil/s, while using FWR it goes down to around
272 ~13k/s. Using "lock_mode" "none" or "exclusive", the speed is ~52k/s.
273
274 However, this is not something you'll notice or need to worry about
275 unless you're writing near that speed.
276
277 If you need more speed, you can try setting "rotate_probability" which
278 will cause FWR to only check for rotation probabilistically, e.g. if you
279 set this to 0.1 then checks will only be done in about 1 of 10 writes.
280 This can significantly reduce the overhead and increase write speed
281 several times (e.g. 5-8 times), but understand that this will make the
282 writes "overflow" a bit, e.g. file sizes will exceed for a bit if you do
283 size-based rotation. More suitable if you only do size-based rotation
284 since it is usually okay to exceed sizes for a bit.
285
286 I want a filehandle instead of a File::Write::Rotate object!
287 Use Tie::Handle::FileWriteRotate.
288
289 HOMEPAGE
290 Please visit the project's homepage at
291 <https://metacpan.org/release/File-Write-Rotate>.
292
293 SOURCE
294 Source repository is at
295 <https://github.com/perlancar/perl-File-Write-Rotate>.
296
297 BUGS
298 Please report any bugs or feature requests on the bugtracker website
299 <https://rt.cpan.org/Public/Dist/Display.html?Name=File-Write-Rotate>
300
301 When submitting a bug or request, please include a test-file or a patch
302 to an existing test-file that illustrates the bug or desired feature.
303
304 SEE ALSO
305 Log::Dispatch::FileRotate, which inspires this module. Differences
306 between File::Write::Rotate (FWR) and Log::Dispatch::FileRotate (LDFR)
307 are as follows:
308
309 * FWR is not part of the Log::Dispatch family.
310
311 This makes FWR more general to use.
312
313 For using together with Log::Dispatch/Log4perl, I have also written
314 Log::Dispatch::FileWriteRotate which is a direct (although not a
315 perfect drop-in) replacement for Log::Dispatch::FileRotate.
316
317 * Secondly, FWR does not use Date::Manip.
318
319 Date::Manip is relatively large (loading Date::Manip 6.37 equals to
320 loading 34 files and ~ 22k lines; while FWR itself is only < 1k
321 lines!)
322
323 As a consequence of this, FWR does not support DatePattern; instead,
324 FWR replaces it with a simple daily/monthly/yearly period.
325
326 * And lastly, FWR supports compressing and rotating compressed old
327 files.
328
329 Using separate processes like the Unix logrotate utility means
330 having to deal with yet another race condition. FWR takes care of
331 that for you (see the compress() method). You also have the option
332 to do file compression in the same script/process if you want, which
333 is convenient.
334
335 There is no significant overhead difference between FWR and LDFR (FWR is
336 slightly faster than LDFR on my testing).
337
338 Tie::Handle::FileWriteRotate and Log::Dispatch::FileWriteRotate, which
339 use this module.
340
341 AUTHOR
342 perlancar <[email protected]>
343
344 COPYRIGHT AND LICENSE
345 This software is copyright (c) 2019, 2016, 2015, 2014, 2013, 2012 by
346 [email protected].
347
348 This is free software; you can redistribute it and/or modify it under
349 the same terms as the Perl 5 programming language system itself.
350