CStringIndexator Class Reference

#include <StringIndexator.h>

Inheritance diagram for CStringIndexator:

Collaboration diagram for CStringIndexator:

List of all members.

Public Member Functions

• CStringIndexator ()
• ~CStringIndexator ()
• bool RegisterStringIndices (string IndicesStr)

  read index declarations from a string and register them
  

• void SetPath (string Path)

  set the path to the indices
  

• string GetIndicesString () const

  return all registered index declarations
  

• CStringIndexSet * GetIndexByNameOrShortName (const string &Name)

  return a pointer to the index by CStringIndexSet::m_Name or CStringIndexSet::m_ShortName
  

• size_t GetSearchPeriodsCount () const

  return the number of corpus periods
  

• const CTokenNo & GetSearchPeriod (size_t i) const

  get a corpus period by an index
  

• bool StartIndexing (string Path)

  call CreateTempFiles for all registered indices
  

• bool TerminateIndexing ()

  call DeleteTempFiles for all registered indices
  

• bool FinalSaveAllIndices (bool bAfterLoading)

  final saving all indices to disk (converting temp files to persistent)
  

• bool AddInputLoadIndexToMemoryLoadIndex ()

  unites input index with memory index and clears input load index
  

• bool AddMemoryLoadIndexToMainLoadIndex ()

  unites memory index with main index and clears memory load index
  

• bool SaveMemoryLoadIndex ()

  store memory load index on the disk
  

• CStringIndexSet * GetIndexByName (const string &Name)

  return a pointer to the index by CStringIndexSet::m_Name
  

• CStringIndexSet * GetTokenIndex ()

  return the first index that normally contains tokens themselves
  

• const CStringIndexSet * GetTokenIndex () const

  return the first index that normally contains tokens themselves
  

• void ProcessBigramBorders (const int BreakCollectionNo, CTokenNo occurrence)

  add "Wi <eos>" bigrams for end of sentence
  

Public Attributes

• string m_Path

  where all indices are stored
  

• vector< CStringIndexSet * > m_Indices

  the registered indices
  

• size_t m_MaxRegExpExpansionSize

  the maximal number of index items which can be included in an expansion set of one regular expression
  

• CStringIndexSet * m_pChunkIndex

  a quick reference to a chunk index, if CConcIndexator::m_bIndexChunks is on, otherwise null
  

• CStringIndexSet * m_pLeftBigramsIndex

  a quick reference to the left bigrams index
  

• CStringIndexSet * m_pRightBigramsIndex

  a quick reference to the right bigrams index
  

Protected Member Functions

• bool RegisterChunkIndex ()

  register chunk index (chunks:NP, VP etc)
  

• string GetSearchPeriodsFileName () const

  return the file name for search periods
  

• bool DestroyIndices ()

  call DestroyIndexSet for all registered indices
  

• bool ReadIndicesFromTheDisk ()

  call ReadFromTheDisk for all registered indices
  

• void ClearStringIndices ()

  clear m_Indices
  

• bool IndexOneToken (const char *Line, const CTokenNo &TokenNo)

  index one token and its properies (delimited by CConcCommon.h::globalFieldDelimeter)
  

Protected Attributes

• vector< CTokenNo > m_SearchPeriods

  search periods of the corpus
  

---

Detailed Description

CStringIndexator contains a set of all token indices and corpus periods. It contains also the main path to the project file.

---

Constructor & Destructor Documentation

CStringIndexator::CStringIndexator

(

)

References m_MaxRegExpExpansionSize, m_Path, m_pChunkIndex, m_pLeftBigramsIndex, and m_pRightBigramsIndex.

CStringIndexator::~CStringIndexator

(

)

References ClearStringIndices().

Here is the call graph for this function:

---

Member Function Documentation

bool CStringIndexator::RegisterChunkIndex

(

)

[protected]

register chunk index (chunks:NP, VP etc)

References ChunkIndexName, GetIndexByName(), CStringIndexSet::InitIndexSet(), m_Indices, and m_pChunkIndex.

Referenced by CConcIndexator::LoadOptionsFromString().

Here is the call graph for this function:

Here is the caller graph for this function:

string CStringIndexator::GetSearchPeriodsFileName

(

)

const [protected]

return the file name for search periods

References m_Path, and MakeFName().

Referenced by CConcIndexator::DestroyIndex(), FinalSaveAllIndices(), ReadIndicesFromTheDisk(), and CConcIndexator::WasIndexed().

Here is the call graph for this function:

Here is the caller graph for this function:

bool CStringIndexator::DestroyIndices

(

)

[protected]

call DestroyIndexSet for all registered indices

References m_Indices.

Referenced by CConcIndexator::DestroyIndex().

Here is the caller graph for this function:

bool CStringIndexator::ReadIndicesFromTheDisk

(

)

[protected]

call ReadFromTheDisk for all registered indices

References GetSearchPeriodsFileName(), m_Indices, m_SearchPeriods, and ReadVector().

Referenced by CConcIndexator::LoadProject().

Here is the call graph for this function:

Here is the caller graph for this function:

void CStringIndexator::ClearStringIndices

(

)

[protected]

clear m_Indices

References m_Indices.

Referenced by CConcIndexator::CreateAsUnion(), RegisterStringIndices(), and ~CStringIndexator().

Here is the caller graph for this function:

bool CStringIndexator::IndexOneToken	(	const char *	Line,
		const CTokenNo &	TokenNo
	)			[protected]

index one token and its properies (delimited by CConcCommon.h::globalFieldDelimeter)

References ddc_archive_stub, ErrorMessage(), Format(), globalFieldDelimeter, and m_Indices.

Referenced by CConcIndexator::IndexMorphXml(), CConcIndexator::IndexOneTableTextArea(), and CConcIndexator::IndexTextOrHtmlFile().

Here is the call graph for this function:

Here is the caller graph for this function:

bool CStringIndexator::RegisterStringIndices

(

string

IndicesStr

)

read index declarations from a string and register them

References ClearStringIndices(), ErrorMessage(), GetIndexByName(), CStringIndexSet::InitIndexSet(), m_Indices, Name, and Trim().

Referenced by CConcIndexator::LoadOptionsFromString().

Here is the call graph for this function:

Here is the caller graph for this function:

void CStringIndexator::SetPath

(

string

Path

)

set the path to the indices

References m_Path.

Referenced by CConcIndexator::LoadSourceFilesAndOptions(), and CConcordance::SaveProject().

Here is the caller graph for this function:

string CStringIndexator::GetIndicesString

(

)

const

return all registered index declarations

References ChunkIndexName, Format(), m_Indices, and Trim().

Referenced by CConcIndexator::CreateAsUnion(), CConcIndexator::LoadOptionsFromString(), and CConcIndexator::SaveOptionsToString().

Here is the call graph for this function:

Here is the caller graph for this function:

CStringIndexSet * CStringIndexator::GetIndexByNameOrShortName

(

const string &

Name

)

return a pointer to the index by CStringIndexSet::m_Name or CStringIndexSet::m_ShortName

References m_Indices.

Referenced by CQueryTokenNode::CreateNodeByIndexName().

Here is the caller graph for this function:

size_t CStringIndexator::GetSearchPeriodsCount

(

)

const

return the number of corpus periods

References m_SearchPeriods.

Referenced by CIndexSetForQueryingStage::AddOccurs(), CIndexSetForQueryingStage::BuildPeriodsDivisionAndCompress(), CStringIndexSet::ConvertLoadIndexToWorkingIndex(), CConcHolder::GetAllHits(), CConcHolder::GetOccurrences(), CIndexSetForQueryingStage::LoadPeriodDevision(), CIndexSetForQueryingStage::ReadAllOccurrences(), CIndexSetForBigrams::ReadAllOccurrences(), CIndexSetForQueryingStage::WritePeriodsDivision(), and CConcIndexatorInvoker::WriteTimeStatistics().

Here is the caller graph for this function:

const CTokenNo& CStringIndexator::GetSearchPeriod

(

size_t

i

)

const [inline]

get a corpus period by an index

References m_SearchPeriods.

Referenced by CIndexSetForQueryingStage::AddOccurs(), CIndexSetForQueryingStage::BuildPeriodsDivisionAndCompress(), and CStringIndexSet::ConvertLoadIndexToWorkingIndex().

Here is the caller graph for this function:

bool CStringIndexator::StartIndexing

(

string

Path

)

call CreateTempFiles for all registered indices

References m_Indices, and m_Path.

Referenced by CConcIndexator::StartIndexing().

Here is the caller graph for this function:

bool CStringIndexator::TerminateIndexing

(

)

call DeleteTempFiles for all registered indices

References m_Indices.

Referenced by CConcIndexator::TerminateIndexing().

Here is the caller graph for this function:

bool CStringIndexator::FinalSaveAllIndices

(

bool

bAfterLoading

)

final saving all indices to disk (converting temp files to persistent)

References GetSearchPeriodsFileName(), m_Indices, m_SearchPeriods, and WriteVector().

Referenced by CConcIndexator::CreateAsUnion(), and CConcIndexatorInvoker::FinalizeIndex().

Here is the call graph for this function:

Here is the caller graph for this function:

bool CStringIndexator::AddInputLoadIndexToMemoryLoadIndex

(

)

unites input index with memory index and clears input load index

References m_Indices.

Referenced by CConcIndexatorInvoker::BuildIndex(), and CConcIndexatorInvoker::FinalizeIndex().

Here is the caller graph for this function:

bool CStringIndexator::AddMemoryLoadIndexToMainLoadIndex

(

)

unites memory index with main index and clears memory load index

References m_Indices.

Referenced by CConcIndexatorInvoker::BuildIndex(), and CConcIndexatorInvoker::FinalizeIndex().

Here is the caller graph for this function:

bool CStringIndexator::SaveMemoryLoadIndex

(

)

store memory load index on the disk

References m_Indices.

Referenced by CConcIndexatorInvoker::BuildIndex(), and CConcIndexatorInvoker::FinalizeIndex().

Here is the caller graph for this function:

CStringIndexSet * CStringIndexator::GetIndexByName

(

const string &

Name

)

return a pointer to the index by CStringIndexSet::m_Name

References m_Indices.

Referenced by CQueryTokenNode::BuildRegExp(), CQueryTokenNode::CreateFileList(), CConcIndexator::CreateMorphIndex(), CQueryTokenNode::CreateTokenPattern(), CQueryTokenNode::EvaluateWithoutHits(), CConcIndexator::LoadOptionsFromString(), RegisterChunkIndex(), and RegisterStringIndices().

Here is the caller graph for this function:

CStringIndexSet * CStringIndexator::GetTokenIndex

(

)

return the first index that normally contains tokens themselves

References m_Indices.

Referenced by CConcHolder::GetFileSnippets(), CConcHolder::GetHitIds(), and CConcHolder::SaveOccurrences().

Here is the caller graph for this function:

const CStringIndexSet * CStringIndexator::GetTokenIndex

(

)

const

return the first index that normally contains tokens themselves

References m_Indices.

void CStringIndexator::ProcessBigramBorders	(	const int	BreakCollectionNo,
		CTokenNo	occurrence
	)

add "Wi <eos>" bigrams for end of sentence

References m_Indices.

Referenced by CConcIndexator::IndexOneTableTextArea(), and CConcIndexator::IndexTextOrHtmlFile().

Here is the caller graph for this function:

---

Member Data Documentation

vector<CTokenNo> CStringIndexator::m_SearchPeriods [protected]

search periods of the corpus

Referenced by CConcIndexator::CalculateSearchPeriods(), FinalSaveAllIndices(), GetSearchPeriod(), GetSearchPeriodsCount(), and ReadIndicesFromTheDisk().

string CStringIndexator::m_Path

where all indices are stored

Referenced by CIndexSetForQueryingStage::AssertHasPath(), CConcIndexator::AssertHasPath(), CConcIndexatorInvoker::BuildIndex(), CConcIndexator::CreateAsUnion(), CConcIndexator::CreateMorphIndex(), CStringIndexator(), CConcIndexator::DestroyIndex(), CConcIndexatorInvoker::FinalizeIndex(), CConcIndexator::GetFileNameForCorpusFileNames(), CIndexSetForQueryingStage::GetFileNameForInfos(), CConcIndexator::GetFileNameForMaskedFiles(), CStringIndexSet::GetLeftBigramsFileName(), CIndexSetForQueryingStage::GetOccHdrFileName(), CIndexSetForQueryingStage::GetOccursFileName(), CIndexSetForQueryingStage::GetPeriodsDevisionFileName(), CIndexSetForBigrams::GetRightToLeftPerdiv(), GetSearchPeriodsFileName(), CStringIndexSet::GetStorageFileName(), CConcIndexator::InitDefaultOptions(), CConcIndexator::LoadCorpusFiles(), CConcIndexator::LoadProject(), CConcIndexator::LoadSourceFilesAndOptions(), SetPath(), StartIndexing(), CConcIndexator::StartIndexing(), CConcIndexator::TerminateIndexing(), CConcIndexator::WasIndexed(), and CConcIndexatorInvoker::WriteTimeStatistics().

vector<CStringIndexSet*> CStringIndexator::m_Indices

the registered indices

Referenced by AddInputLoadIndexToMemoryLoadIndex(), AddMemoryLoadIndexToMainLoadIndex(), CConcHolder::BuildJsonContextString(), ClearStringIndices(), CConcIndexator::CreateAsUnion(), DestroyIndices(), FinalSaveAllIndices(), CConcHolder::GenerateOneHitStringJson(), GetIndexByName(), GetIndexByNameOrShortName(), GetIndicesString(), GetTokenIndex(), CConcHolder::GetTokensFromStorageByBreak(), IndexOneToken(), CConcIndexator::LoadOptionsFromString(), ProcessBigramBorders(), ReadIndicesFromTheDisk(), RegisterChunkIndex(), RegisterStringIndices(), SaveMemoryLoadIndex(), CConcHolder::SaveOccurrences(), CConcIndexator::SaveOptionsToString(), StartIndexing(), and TerminateIndexing().

size_t CStringIndexator::m_MaxRegExpExpansionSize

the maximal number of index items which can be included in an expansion set of one regular expression

Referenced by CStringIndexator(), CConcIndexator::LoadOptionsFromString(), CStringIndexSet::QueryTokenList(), CStringIndexSet::QueryTokenListUsingRegExp(), CStringIndexSet::QueryTokenListWithRightTruncation(), and CConcIndexator::SaveOptionsToString().

CStringIndexSet* CStringIndexator::m_pChunkIndex

a quick reference to a chunk index, if CConcIndexator::m_bIndexChunks is on, otherwise null

Referenced by CQueryTokenNode::CreateChunkPattern(), CStringIndexator(), CQueryTokenNode::EvaluateWithoutHits(), CConcIndexator::IndexOneTableTextArea(), and RegisterChunkIndex().

CStringIndexSet* CStringIndexator::m_pLeftBigramsIndex

a quick reference to the left bigrams index

Referenced by CStringIndexator().

CStringIndexSet* CStringIndexator::m_pRightBigramsIndex

a quick reference to the right bigrams index

Referenced by CStringIndexator().

---

The documentation for this class was generated from the following files:

• StringIndexator.h
• StringIndexator.cpp