TBFILT(1) Retro Project Manual TBFILT(1) NNAAMMEE tbfilt - filter for and analysis of test bench log files SSYYNNOOPPSSIISS ttbbffiilltt ----tteeee _O_F_I_L_E [_O_P_T_I_O_N]... ttbbffiilltt [_O_P_T_I_O_N]... [_F_I_L_E_S]... DDEESSCCRRIIPPTTIIOONN PPrriinncciippllee ooff OOppeerraattiioonn ttbbffiilltt is the central tool to analyze the log files created by the test benches. It scans the test bench output for messages which indicate a test failure and based on this marks a test as PPAASSSSed or FFAAIILLed. It can be used in two modes: - as a filter during test bench execution, typically in a setup like tbw 2>&1 | tbfilt --tee= tbfilt reads the output from the test bench via _s_t_d_i_n and a pipe, filters out the messages indicating a failure and shows them on _s_t_d_o_u_t, and saves the full test bench output to the file given in the ----tteeee option. In this mode tbfilt works similar to a tee ... | egrep ... pipeline with a very involved egrep selection expression. The exit status of tbfilt is 1 in case the test is considered as FFAAIILLed. - as log file analysis tool. In this case, the test bench log files are either specified explicitly as arguments or determined via the ----ffiinndd or ----aallll options. If the ----ssuummmmaarryy option is specified a one-line summary for each test log file is displayed. The format of this summary is configurable via the ----ffoorrmmaatt, ----wwiiddee, and ----ccoomm-- ppaacctt options and via the TTBBFFIILLTT__FFOORRMMAATT environment variable. The exit status of tbfilt is 1 in case any of the tests is considered as FFAAIILLed. FFiilltteerr CCrriitteerriiaa A line that contains any of the following strings is considered an in- dication of a FFAAIILLed test: --EE:: --FF:: EERRRROORR FFAAIILL ::((aasssseerrttiioonn wwaarrnniinngg)):: ::((aasssseerrttiioonn eerrrroorr)):: ::((aasssseerrttiioonn ffaaiilluurree)):: As an exception to the general rules above the following assertion mes- sages are accepted: Assertion warnings from IEEE libraries at startup (t=0ns) are ignored. They are hard to avoid in complex models and gen- erally don't indicate a real issue. - assertion warnings from IEEE libraries at startup (t=0ns). They are hard to avoid in complex models and in general, don't indicate a real issue. The best is to suppress them in gghhddll(1) with the option '--ieee-asserts=disable-at-0'. That's usually done via the TTBBWW__GGHHDDLL__OOPPTTSS environment variable used by ttbbww(1). - assertion failure with the text 'Simulation Finished'. It is used to end simulations in gghhddll(1) in some test benches. tbfilt also expects a line in one of the formats xxx ns: DONE -- tb'swithout clock xxx.x ns xxx: DONE -- single clock tb's xxx.x ns xxx: DONE-xxx -- multiclock tb's (max taken) and considers a test FFAAIILLed if it is missing. In addition, lines containing --WW:: PPAASSSS will be displayed. If the ----ppccoomm option is specified also all lines starting with 'C'. Finally, tbfilt checks for a line of the format real xmx.xxxs user xmx.xxxs sys xmx.xxxs and extracts the test bench execution times from this. It can be gener- ated by a bbaasshh(1) 'time' command when export TIMEFORMAT=$'real %3lR user %3lU sys %3lS' is set. The wrapper scripts bbrruunn__ttbbww(1) or ttbbrruunn__ttbbwwrrrrii(1) are in gen- eral used to set this up correctly. OOPPTTIIOONNSS FFiilltteerr OOppttiioonnss ----tteeee==_o_f_i_l_e All log file input is written to _o_f_i_l_e. Typically used to save the test bench output in a file when tbfilt is used in filter mode in a pipeline and reads from _s_t_d_i_n. ----ppccoomm Enables that lines starting with "C" are also displayed. FFiillee SSeelleeccttiioonn OOppttiioonnss ----ffiinndd==_p_a_t_t When given the input files are determined with a ffiinndd(1) com- mand. The selection pattern _p_a_t_t is used with a find -regex in egrep mode. This is functionally similar to a find -regextype egrep -regex '_p_a_t_t' | sort | tbfilt .... pipeline. When no '*' wildcard is found in _p_a_t_t it is assumed to be a mode specification and the pattern is prefixed by .*/tb_.*_ and suffixed by .*\.log to select all log files of a given mode (e.g. 'bsim'). ----aallll When given uses as input files all test bench files which con- form the naming convention. Is equivalent to the option --find '.*/tb_.*_[bfsorept]sim(_.*)?\.log' SSuummmmaarryy OOppttiioonnss ----ssuummmmaarryy Selects summary mode. Only a single summary line per input file is written. The format is configurable via the ----ffoorrmmaatt, ----wwiiddee, and ----ccoommppaacctt options and via the TTBBFFIILLTT__FFOORRMMAATT environ- ment variable. The precedence is (in increasing priority): - build default ('%ec %pf %nf') - TTBBFFIILLTT__FFOORRMMAATT option - ----wwiiddee option - ----ccoommppaacctt option - ----ffoorrmmaatt option ----wwiiddee Selects a wide format for summary outputs, designed to give the most pertinent information. Uses a format of "%fd %fs %tr %tc %sc %ec %pf %nf". ----ccoommppaacctt Selects a compact format for summary outputs, designed to give the key info on an 80 character wide line. Uses a format of "%fa %tg %sg %ec %pf %ns". ----nnoohheeaadd Suppresses the headline of summary outputs. Useful if the sum- mary output is piped into sort or other tools. ----ffoorrmmaatt==_f_m_t Defined the format of the summary lines. The format specifica- tion _f_m_t string is a sequence of conversion specifications of the form '%xx', which will be replaced by the respective values and other characters which are simply copied (usually a blank as delimiter). The supported conversion specifications are: %%ffdd modification date of input file (as yyyy-mm-dd) %%fftt modification time of input file (as hh:mm:ss) %%ffss modification time of input file short format (as hh:mm) %%ffaa age of input file in seconds, minutes, hours or days %%ttrr real (wall clock) time of test bench run %%ttuu user time of test bench run %%ttss system time of test bench run %%ttcc total cpu (user+system) time of test bench run %%ttgg show '%tc c' if cpu time is significant, otherwise '%tr r' %%sstt simulation time in ns %%ssss simulation time short format (in usec, msec, or sec) %%sscc main system clock cycles till DONE %%ssgg use %sc, if available, otherwise %ss %%sspp cpu time per simulation clock cycle (in usec or msec) %%ssmm estimate of system clock rate (in MHz) %%eecc error count %%ppff PASS or FAIL, derived from error count %%nnff full file name (with path) %%nnss short file name (without path) EEXXIITT SSTTAATTUUSS In case of an error, the exit codes are: 11 run time errors, like a bad option, file not found, etc 22 at least one test FAILed, test bench has FAILed EENNVVIIRROONNMMEENNTT TTBBFFIILLTT__FFOORRMMAATT Defines the default summary format and overwrites the build-in de- fault of "%ec %pf %nf". TTBBWW__GGHHDDLL__OOPPTTSS Used by ttbbww(1) to define additional options for gghhlldd-based simula- tions. Usually set to "--unbuffered --ieee-asserts=disable-at-0", see ttbbww(1) documentation. EEXXAAMMPPLLEESS AAss FFiilltteerr Usually used together with ttbbww(1) in pipelines like tbw 2>&1 | tbfilt --tee= Since tbfilt expects also the output of a bbaasshh(1) 'time' command in the input stream the setup of the pipeline is more involved. In general the wrapper scripts ttbbrruunn__ttbbww(1) or ttbbrruunn__ttbbwwrrrrii(1) are used. AAss AAnnaallyyssiiss TTooooll To generate a compact overview of all test bench outputs use cd $RETROBASE tbfilt -all -summary -compact To generate a report indicating all FFAAIILLed test use cd $RETROBASE tbfilt -all -summary -nohead | grep FAIL SSEEEE AALLSSOO ttbbww(1), ttbbrruunn__ttbbww(1), ttbbrruunn__ttbbwwrrrrii(1) AAUUTTHHOORR Walter F.J. Mueller Retro Project 2022-05-02 TBFILT(1)