Ninja: DepsLog Struct Reference

Ninja

As build commands run they can output extra dependency information (e.g. More...

#include <deps_log.h>

List of all members.

Classes
struct	Deps
Public Member Functions
void	Close ()
const vector< Deps * > &	deps () const
	DepsLog ()
Deps *	GetDeps (Node *node)
bool	Load (const string &path, State state, string err)
const vector< Node * > &	nodes () const
	Used for tests.
bool	OpenForWrite (const string &path, string *err)
bool	Recompact (const string &path, string *err)
	Rewrite the known log entries, throwing away old data.
bool	RecordDeps (Node node, TimeStamp mtime, const vector< Node > &nodes)
bool	RecordDeps (Node node, TimeStamp mtime, int node_count, Node *nodes)
	~DepsLog ()
Private Member Functions
bool	RecordId (Node *node)
bool	UpdateDeps (int out_id, Deps *deps)
Private Attributes
vector< Deps * >	deps_
	Maps id -> deps of that id.
FILE *	file_
bool	needs_recompaction_
vector< Node * >	nodes_
	Maps id -> Node.
Friends
struct	DepsLogTest

Detailed Description

As build commands run they can output extra dependency information (e.g.

header dependencies for C source) dynamically. DepsLog collects that information at build time and uses it for subsequent builds.

The on-disk format is based on two primary design constraints:

it must be written to as a stream (during the build, which may be interrupted);
it can be read all at once on startup. (Alternative designs, where it contains indexing information, were considered and discarded as too complicated to implement; if the file is small than reading it fully on startup is acceptable.) Here are some stats from the Windows Chrome dependency files, to help guide the design space. The total text in the files sums to 90mb so some compression is warranted to keep load-time fast. There's about 10k files worth of dependencies that reference about 40k total paths totalling 2mb of unique strings.

Based on these stats, here's the current design. The file is structured as version header followed by a sequence of records. Each record is either a path string or a dependency list. Numbering the path strings in file order gives them dense integer ids. A dependency list maps an output id to a list of input ids.

Concretely, a record is: two bytes record length, high bit indicates record type (implies max record length 32k) path records contain just the string name of the path dependency records are an array of 4-byte integers [output path id, output path mtime, input path id, input path id...] (The mtime is compared against the on-disk output path mtime to verify the stored data is up-to-date.) If two records reference the same output the latter one in the file wins, allowing updates to just be appended to the file. A separate repacking step can run occasionally to remove dead records.

Definition at line 63 of file deps_log.h.

Constructor & Destructor Documentation

DepsLog::DepsLog ( ) [inline]

Definition at line 64 of file deps_log.h.

DepsLog::~DepsLog ( )

Definition at line 35 of file deps_log.cc.

References Close().

Member Function Documentation

void DepsLog::Close ( )

Definition at line 137 of file deps_log.cc.

References file_.

Referenced by OpenForWrite(), TEST_F(), and ~DepsLog().

const vector<Deps*>& DepsLog::deps ( ) const [inline]

Definition at line 90 of file deps_log.h.

Referenced by Load(), Recompact(), RecordDeps(), and UpdateDeps().

DepsLog::Deps * DepsLog::GetDeps ( Node * node )

Definition at line 248 of file deps_log.cc.

References deps_, and Node::id().

Referenced by ImplicitDepLoader::LoadDepsFromLog(), and RecordDeps().

bool DepsLog::Load	(	const string &	path,
		State *	state,
		string *	err
	)

Definition at line 143 of file deps_log.cc.

References deps(), State::GetNode(), Node::id(), kCurrentVersion, kFileSignature, METRIC_RECORD, needs_recompaction_, DepsLog::Deps::nodes, nodes_, Node::set_id(), Truncate(), and UpdateDeps().

Referenced by TEST_F().

const vector<Node*>& DepsLog::nodes ( ) const [inline]

Used for tests.

Definition at line 89 of file deps_log.h.

bool DepsLog::OpenForWrite	(	const string &	path,
		string *	err
	)

Definition at line 39 of file deps_log.cc.

References Close(), file_, kCurrentVersion, kFileSignature, needs_recompaction_, Recompact(), and SetCloseOnExec().

Referenced by TEST_F().

bool DepsLog::Recompact	(	const string &	path,
		string *	err
	)

Rewrite the known log entries, throwing away old data.

Definition at line 256 of file deps_log.cc.

References deps(), deps_, METRIC_RECORD, DepsLog::Deps::mtime, DepsLog::Deps::node_count, DepsLog::Deps::nodes, and nodes_.

Referenced by OpenForWrite().

bool DepsLog::RecordDeps	(	Node *	node,
		TimeStamp	mtime,
		const vector< Node * > &	nodes
	)

Definition at line 71 of file deps_log.cc.

Referenced by Builder::FinishCommand().

bool DepsLog::RecordDeps	(	Node *	node,
		TimeStamp	mtime,
		int	node_count,
		Node **	nodes
	)

Definition at line 77 of file deps_log.cc.

References deps(), file_, GetDeps(), Node::id(), DepsLog::Deps::mtime, DepsLog::Deps::node_count, DepsLog::Deps::nodes, RecordId(), and UpdateDeps().

bool DepsLog::RecordId ( Node * node ) [private]

Definition at line 317 of file deps_log.cc.

References file_, nodes_, Node::path(), and Node::set_id().

Referenced by RecordDeps().

bool DepsLog::UpdateDeps	(	int	out_id,
		Deps *	deps
	)		`[private]`

Definition at line 306 of file deps_log.cc.

References deps(), and deps_.

Referenced by Load(), and RecordDeps().

Friends And Related Function Documentation

friend struct DepsLogTest [friend]

Definition at line 107 of file deps_log.h.

Member Data Documentation

vector<Deps*> DepsLog::deps_ [private]

Maps id -> deps of that id.

Definition at line 105 of file deps_log.h.

Referenced by GetDeps(), Recompact(), and UpdateDeps().

FILE* DepsLog::file_ [private]

Definition at line 100 of file deps_log.h.

Referenced by Close(), OpenForWrite(), RecordDeps(), and RecordId().

bool DepsLog::needs_recompaction_ [private]

Definition at line 99 of file deps_log.h.

Referenced by Load(), and OpenForWrite().

vector<Node*> DepsLog::nodes_ [private]

Maps id -> Node.

Definition at line 103 of file deps_log.h.

Referenced by Load(), Recompact(), and RecordId().

The documentation for this struct was generated from the following files: