Index

lognestmonster / 9046b13

A general-purpose single-header C logging library and parser for event-based logs. (Incomplete)

Download | Refs | Commit Log | Files | README | LICENSE

Latest Commit

{#}TimeHashSubjectAuthor#(+)(-)GPG?
4226 Dec 2018 13:25ece9cdeUpdate README.mdJoshua13032N

Blob @ lognestmonster / README.md

text/plain15060 bytesdownload raw
1<img src="/static/logo.png" height="200px"/>
2
3_Advanced node.js logging for advanced programs._
4
5# log-nest-monster
6Most loggers available only use a _linear_ method of logging; there are verbosity levels and tags to narrow searches, but everything is still on the same level or plane nonetheless. The package `lognestmonster` is a similar type of logger, but it allows you to create multiple **layers** (or "**nests**") of log statements. This is useful because, although you may have to put in some extra organizational work on the code side, the log results are much more clean and thorough, boosting workflow efficiency. The user has absolute control over the way log statements are pushed to their log files.
7
8_Why should I use this?_ There are many projects that create insane amounts of data, almost impossible to sift through without a helper program. The purpose of this logging system is to be a time-saver. Although it takes more time to put it in place, it's almost immediately made up with the performance gain through using the new log format. One thing that's unique about this logger compared to others is that it allows multiple queues to be made, in turn allowing you to split up your data. For example, a Node web server could keep one log file that records everything that the backend does while it uses another to record user or traffic information for analytics. Parsing software could be used to read either one.
9
10## Classes
11
12There are 4 classes offered by the package: `Logger`, the driving device that organizes everything; `Queue`, the class that actually pushes the log statements to their respective file; `Statement`, the actual log data (timestamp, verbosity, tag/invoker, message); and `Event`, a nest layer for `Statement`s or other `Event`s.
13
14The following subsections assume that `lognestmonster` has been `require`d by node.js with the following code:
15```javascript
16const Logger = require("lognestmonster");
17```
18
19### Logger
20
21Creates and organizes your queues. `Logger` is the exported package. `Logger.Logger` is the Logger class.
22
23Logger.Logger()
24```javascript
25var MyLogger = new Logger.Logger(Object config);
26```
27Where `Object config` defaults to:
28```json
29{"name": "Logger", "locations": {"node": "./log/node"}}
30```
31`name` serves no purpose other than identification. `locations` is used to create new queues, taking each key as the queue name and each value as the queue output location.
32
33Logger.Logger.queue()
34```javascript
35MyLogger.queue(string name);
36```
37This returns the appropriate `Queue` object for the provided `name`, assuming it was created with the `Logger` object.
38
39### Queue
40
41Manages log `Statement`s and `Event`s and how they're written to the final log file. Note that these are implicitly created with `Logger.Logger` when the `locations` object is properly provided in the `config` parameter of the `Logger.Logger` constructor.
42
43Logger.Queue()
44```javascript
45let MyQueue = new Logger.Queue(string name, string location);
46// note that parameters are the same format as key-value
47// pairs in `config.locations` of `Logger.Logger(config)`
48```
49This creates the queue, taking `name` to be used as its ID and `location` as the path to where the log file should be created.
50
51Logger.Queue.push()
52```javascript
53MyQueue.push(Statement statement);
54MyQueue.push(Event event);
55MyQueue.push(string verbosity, string tag, string message); // implicitly creates a `Statement` object
56```
57This adds log items or nests to the to-write queue. This returns the Queue object.
58
59Logger.Queue.write()
60```javascript
61MyQueue.write();
62```
63This appends every queue value to the log file, emptying the queue. This returns the Queue object.
64
65### Statement
66
67This is the base log item where written log data is actually held.
68
69Logger.Statement()
70```javascript
71let MyStatement = Logger.Statement(string verbosity, string tag, string message);
72```
73
74The timestamp value is created automatically by the constructor.
75
76### Event
77
78This is the proper name for a nest. Essentially, it's just an array that can hold other `Event` objects and `Statement` objects, creating a tree.
79
80Logger.Event()
81```javascript
82let MyEvent = Logger.Event();
83let MyEvent = Logger.Event(Event event);
84let MyEvent = Logger.Event(Statement statement);
85let MyEvent = Logger.Event(string verbosity, string tag, string message);
86```
87Any arguments given are passed to `this.push()`.
88
89Logger.Event.push()
90```javascript
91MyEvent.push(Event event);
92MyEvent.push(Statement statement);
93MyEvent.push(string verbosity, string tag, string message);
94```
95This message pushes an `Event` or `Statement` as items in the nest. In the case that 3 strings are given as arguments, a `Statement` is implicitly created. This returns the Event object.
96
97## Verbosity Levels
98
99When creating a `Statement`, you can pass anything you'd like for the first `verbosity` string, although there are some ones preset by the package:
100
101`Logger.INFO`
102`Logger.DEBUG`
103`Logger.VERBOSE`
104`Logger.VERYVERBOSE`
105`Logger.WARNING`
106`Logger.ERROR`
107
108These verbosity levels can be used to narrow down your search results when parsing the log files.
109
110## Sample Usage
111
112Here's an example of how somebody would initiate the Logger, create and push items to the Queue, and write them to the log file. **PLEASE SEE THE NOTES ABOUT THIS EXAMPLE DOWN BELOW.**
113```javascript
114// Require the package
115const Logger = require("lognestmonster");
116
117// Creates the logger object. Placing the new Logger inside the package allows cross-file usage, so you only have to initiate once.
118Logger.Overseer = new Logger.Logger({
119 name: "Overseer",
120 locations: {
121 "node": "./log/node" // Creates a queue named `node` that uses the path `./log/node`
122 },
123});
124
125// Pushes a statement directly to the `node` queue
126Logger.Overseer.queue("node").push(Logger.INFO, "PROCESS", "Process started");
127
128// Creates a new event and pushes a Statement to it
129let LoadEvent = new Logger.Event();
130LoadEvent.push(Logger.INFO, "INIT", "Acquiring needed top-level packages.");
131
132// Creates a new event and pushes multiple Statements to it
133let LowerNestedEvent = new Logger.Event();
134
135LowerNestedEvent.push(Logger.INFO, "INIT", "Loading fs...");
136LowerNestedEvent.push(Logger.DEBUG, "INIT", "fs loaded.");
137
138LowerNestedEvent.push(Logger.INFO, "INIT", "Loading http...");
139LowerNestedEvent.push(Logger.DEBUG, "INIT", "http loaded.");
140
141LowerNestedEvent.push(Logger.INFO, "INIT", "Loading jsonwebtoken...");
142LowerNestedEvent.push(Logger.DEBUG, "INIT", "jsonwebtoken loaded.");
143
144LowerNestedEvent.push(Logger.INFO, "INIT", "Loading lognestmonster...");
145LowerNestedEvent.push(Logger.DEBUG, "INIT", "lognestmonster loaded.");
146
147// Pushes the Event LowerNestedEvent to the Event LoadEvent
148LoadEvent.push(LowerNestedEvent);
149
150// Pushes another statement to LoadEvent
151LoadEvent.push(Logger.INFO, "INIT", "Finished.");
152
153// Pushes LoadEvent (a nest that consists of [Statement, Event, Statement] now) to the write queue
154Logger.Overseer.queue("node").push(LoadEvent);
155
156// Queue should now look like this: [Statement, Event]
157
158// Writes the queue, effectively emptying it into the log file
159Logger.Overseer.queue("node").write();
160```
161
162The above code creates the following JSON-like log output in the designated file:
163```json
164{"timestamp":"2018-12-26T18:08:37.654Z","verbosity":"INFO","tag":"PROCESS","message":"Process started"}
165[{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Acquiring needed top-level packages."},[{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Loading fs..."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"DEBUG","tag":"INIT","message":"fs loaded."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Loading http..."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"DEBUG","tag":"INIT","message":"http loaded."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Loading jsonwebtoken..."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"DEBUG","tag":"INIT","message":"jsonwebtoken loaded."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Loading lognestmonster..."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"DEBUG","tag":"INIT","message":"lognestmonster loaded."}],{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Finished."}]
166```
167
168_To see how to parse this data (putting it into proper JSON), see the Log Format section._
169
170### Notes about this example
171
172_Large projects_
173If your project spans multiple files, you could easily place your Logger object into the package itself, allowing the same Logger object to be accessed by other parts of your project. This is seen in the sample code with `Logger.Overseer = new Logger.Logger(...)`.
174
175_Repetition_
176There is some repetitive code; specifically, `Logger.Overseer.queue("node")`. Do note that this actually results in a `Queue` object, so you could easily make this its own variable like this:
177```javascript
178let NodeQueue = Logger.Overseer.queue("node");
179NodeQueue.push(...).write();
180```
181
182_Queue pushing and writing_
183When you're pushing multiple objects to a queue, be wary that they will stay there until the queue is written. Because everything is its own class, what you're really pushing is a _reference_ to the real object, so you can make changes to a pushed object after it has been pushed. As such, it is entirely possible to prematurely write a queue before an already-pushed `Event` or `Statement` is finished. It's good practice to immediately write the queue immediately after it has been pushed. Perhaps in the future there will be some functionality where a `Statement` or `Event` can be directly written instead of placed in the queue.
184
185_Push order_
186The developer has control over **everything** here. The order of your log file is the order that you push to your `Event`s and `Queue`. If I were to push a new Statement to `LoadEvent` in the middle of the pushes to `LowerNestedEvent`, it would show up on the log first because `LowerNestedEvent` isn't itself pushed to `LoadEvent` until later in the code. The order of the log file is 100% logical and in the developer's control, so it would be wise to write your code neatly with this in mind. This logging package doesn't do anything you don't tell it to.
187
188## Log Format
189
190Logs are controlled by `Queue` objects. Placed in the folder they're told, the logs should follow the ISO datetime format and have a `.log` file extension. Here's the name of a sample log file: `2018-12-26T18-08-37-653Z.log`
191
192Regarding format, logs follow a JSON-like format, as follows:
193```json
194{"timestamp":"2018-12-26T18:08:37.654Z","verbosity":"INFO","tag":"PROCESS","message":"Process started"}
195[{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Acquiring needed top-level packages."},[{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Loading fs..."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"DEBUG","tag":"INIT","message":"fs loaded."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Loading http..."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"DEBUG","tag":"INIT","message":"http loaded."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Loading jsonwebtoken..."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"DEBUG","tag":"INIT","message":"jsonwebtoken loaded."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Loading lognestmonster..."},{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"DEBUG","tag":"INIT","message":"lognestmonster loaded."}],{"timestamp":"2018-12-26T18:08:37.655Z","verbosity":"INFO","tag":"INIT","message":"Finished."}]
196```
197
198Because of the way the Queue objects push it (the most efficient way regarding computing power with changing/appending to files), you will have to do a bit of tweaking to get it in proper JSON format:
199
2001. Add a comma before every newline (except for the last one at the end of the file)
2012. Wrap everything in table brackets ('[' and ']')
202
203This could be easily automated. Once finished, you get proper JSON (below), where each object `{}` is a Statement and each wrapping table `[]` is an Event (excluding the outermost one).
204
205```json
206[
207 {
208 "timestamp": "2018-12-26T05:55:23.360Z",
209 "verbosity": "INFO",
210 "tag": "PROCESS",
211 "message": "Process started"
212 },
213 [
214 {
215 "timestamp": "2018-12-26T05:55:23.360Z",
216 "verbosity": "INFO",
217 "tag": "INIT",
218 "message": "Acquiring needed top-level packages."
219 },
220 [
221 {
222 "timestamp": "2018-12-26T05:55:23.360Z",
223 "verbosity": "INFO",
224 "tag": "INIT",
225 "message": "Loading fs..."
226 },
227 {
228 "timestamp": "2018-12-26T05:55:23.360Z",
229 "verbosity": "DEBUG",
230 "tag": "INIT",
231 "message": "fs loaded."
232 },
233 {
234 "timestamp": "2018-12-26T05:55:23.360Z",
235 "verbosity": "INFO",
236 "tag": "INIT",
237 "message": "Loading http..."
238 },
239 {
240 "timestamp": "2018-12-26T05:55:23.360Z",
241 "verbosity": "DEBUG",
242 "tag": "INIT",
243 "message": "http loaded."
244 },
245 {
246 "timestamp": "2018-12-26T05:55:23.360Z",
247 "verbosity": "INFO",
248 "tag": "INIT",
249 "message": "Loading jsonwebtoken..."
250 },
251 {
252 "timestamp": "2018-12-26T05:55:23.360Z",
253 "verbosity": "DEBUG",
254 "tag": "INIT",
255 "message": "jsonwebtoken loaded."
256 },
257 {
258 "timestamp": "2018-12-26T05:55:23.360Z",
259 "verbosity": "INFO",
260 "tag": "INIT",
261 "message": "Loading lognestmonster..."
262 },
263 {
264 "timestamp": "2018-12-26T05:55:23.360Z",
265 "verbosity": "DEBUG",
266 "tag": "INIT",
267 "message": "lognestmonster loaded."
268 }
269 ],
270 {
271 "timestamp": "2018-12-26T05:55:23.360Z",
272 "verbosity": "INFO",
273 "tag": "INIT",
274 "message": "Finished."
275 }
276 ]
277]
278```
279
280The above can be taken in by parsing software and create something similar to the following:
281
282```
283[[LOG FILE NAME]]
284[[LOG DATE]]
285[[LOG FILE SIZE]]
286[[LOG ITEM COUNT]]
287
288TIMESTAMP - INFO - PROCESS - Process started
289v 3 ITEMS
290 TIMESTAMP - INFO - INIT - Acquiring needed top-level packages.
291 v 6 ITEMS
292 TIMESTAMP - INFO - INIT - Loading fs...
293 TIMESTAMP - DEBUG - INIT - fs loaded.
294 TIMESTAMP - INFO - INIT - Loading http...
295 TIMESTAMP - DEBUG - INIT - http loaded.
296 TIMESTAMP - INFO - INIT - Loading jsonwebtoken...
297 TIMESTAMP - DEBUG - INIT - jsonwebtoken loaded.
298 TIMESTAMP - INFO - INIT - Finished.
299```
300
301This is exciting! You still have the ability to narrow down your results with timestamps, verbosity levels, and tags/invokers, but now you have the organization with collapsable nests to not be overwhelmed with large amounts of data at the same time.
302
303## Contributors
304
305Developer - Joshua 'joshuas3' Stockin \<joshstockin@gmail.com\> (https://www.github.com/joshuas3)
306
307Name - Patrik 'Patrola' Xop (https://github.com/PatrikXop)
308