TOP 10 VERIFICATION MYTHS

1. This is legacy code no need to verify it - Hold your horses! Are you 100% sure
that you’re dealing with silicon proven code? Are you sure that nobody’s touched
it since it last worked?
2. I can come up with a patch in 5 minutes - That’s OK as long as you make sure
you don’t end up with a verification environment that looks like a bunch of
patches hooked together. Ask yourself how easily it would be to modify or fix your
environment a week from today? Isn’t it worth it to take a couple of more minutes
and write a robust code?
3. Start checking and plan as you go - A big no no! Don’t be tempted into this.
Even if your task looks like a piece of cake, always plan in advance. You’ll be
amazed to see how many problems can be completely avoided. Remember the 5
P's: Proper Planning Prevents Poor Performance.
4. This is really simple, no need for a test plan - Consider your test plan document
as your working contract. Whatever you put in it defines the requirements for
your current work. If it’s a simple task then just write a test plan on half a page.
5. Verification is not the product, no need to keep software standards - True,
verification is not the product but still - when you’re dealing with thousands lines
of code, you’d better make sure there is a certain level of consistency, let alone
programming errors.
6. Don’t have time to add comments - Remember the last time you spent half a day
on reverse engineering somebody else’s code? How about your own code? Better
yet, start each test with comments explaining the steps in the test and keep them
up to date.
7. Oh I know! Let’s just force the signal from outside - Forced wires have this
tendency to get forgotten along the way and then reappear at a later stage,
usually a week before tapeout. So be extra careful.
8. Must have regression running in the background all the time - Regression
runs alone can’t do the job. You must have a Regression Sitter to monitor and
analyze the results or else - you’re just wearing out your servers.
9. We have reached 100% coverage there’s no point in running more tests - Not
really. Your coverage model can only capture what you thought about in advance.
Obviously a random test bench is capable of generating additional scenarios that
might reveal a bug, so don’t stop at 100%. Instead - enhance the functional
coverage model.
10. Verifiers should be looking for bugs - This is a common misconception of what
verification is all about. Verifiers should be focused on building a well
constructed, robust and complete test bench. Bugs will come out on their own.

Functional Coverage vs Code Coverage

Code coverage is a measure of what parts of the RTL implementation were executed by your simulator while running the testcases.

Functional coverage is a measure of the level of Functionality of the RTL covered by the testcases. Unlike code coverage the metric in Functional coverage i.e. the Functionality is defined by us using functional coverage groups. There are various technologies available to define these functional coverage points and to know if they were reached.

Both metrics are good and give different information about verification suite's quality.

Various flavors of code coverage metric tells us about how good is the Stimulus to the Design Under Test.For e.g some lines are not covered or some signals are not toggled, it means that the testbench and testcases are not good enough to make the design reach these states.
Code coverage cannot tell us any relation between to different pieces of rtl code. E.g. Whether two signals toggled together...or whether read-logic of block A and write logic of block B was excited at the same time? .It can only tell us if a signal ever toggled or if a logic code was ever reached.

Functional coverage points are an indicator coverage of the design's functional state. A functional state can be achieved by combination of different pieces of code or different signals. So its is a bit stronger metric to measure verification completion. But by definition the functional coverage metric is very subjective. The goodness of a functional coverage report is only as good as the functional coverage points and its implementation.

For any coverage metric to hold any meaning, it should be coupled with a good checking mechanism for all the testcases. There is no point in reaching a state of design or exciting a logic and not checking if design responds as expected in that state.


The intent of code and functional coverage differs:

Code Coverage :
1. there is no need to use Spec at all.
2.It verifies test cases completeness in terms of hitting every line, every expression etc of the RTL code.
3.Also, it verifies for the non-accessable (dead) code and some other code-related checks

Functional Coverage :
1. it verifies not only RTL against Spec, but also Spec against higher-level system requirements.
2.Performance verification may reveal functional spec deficiencies as well as deep functional bugs too.

What is verification

Verification is a huge topic and a number of books exist on the subject. As a goal of this blog, treatment of the topic has to be limited to a few aspects only.

Wrong functionality which does not meet the end specification results in products which dont meet customer expectations. Hence verification of the design is needed to make sure that the end specification is met and corrective actions are taken on designs which dont meet them. If verification does not catch a bug in design, wrong designs get out in to the market.

Coverage metrics are defined by most verification engineers. Based on the level of representation, here are a few coverage metrics..

1. Code based metrics (HDL code)
2. Circuit structure based metrics (Netlist)
3. State-space based metrics (State Transition Graphs)
4. Functionality based metrics (User defined Tasks)
5. Spec based metrics (Formal or executable spec)

There are many branches to verification of digital systems.

Below we list a couple of them

1. Simulation (for digital systems)
2. Advanced formal verification of Hardware [equivalence checking, Assertions, Model Checking]
3. Hardware Acceleration (FPGA/Emulation), or hardware/software co-design for simulation..

Simulation aims to verify a given design specification. This is achieved by building a computer model of the hardware being designed and executing the model to analyze it's behavior. For the model to be accurate, it has to include as much information in it as possible to be of any realistic value. At the same time, the model should not consume too much computer memory and operations on the model should not be run time intensive.

There are numerous levels of abstraction at which simulation can be performed..
1. Device level
2. Circuit level
3. Timing and macro level
4. Logic or gate level
5. RTL level
6. Behavioral level

The specification (for the computer model) for a digital system is usually written at a behavioral level or RTL level (we will discuss more about gate level sim later!) :). In addition to the design requirements in the spec, more behavioral or RTL code is written in the form of a wrapper (test bench) around the original design to test and see if the design meets the design intent. The wrapper logic probes the design with functional vectors, collects the responses and verifies them against the expectated response.

A simulator has a kernel to process an input description and apply stimuli on it and represent the result to an end user on a waveform viewer. Internally it creates models for gates, delay, connectivity and numerous other variables.

There are various logic simulators available from numerous CAD vendors. (ModelSim, ncVerilog, VCS). Most of these simulators are a combination of event driven and cycle based mechanisms. They can also handle mixed language designs (VHDL+verilog) and adhere mostly to the Language specification which the IEEE standards committee comes out with. Some of these simulators are mixed mode simulator, i.e they can handle multiple levels of abstraction.

Verification technology has matured over the years. We have many more mechanisms in place apart from simulation.

I will try to list a few of them below and we will cover each one in the future

Detection: Simulation, Lint Tools, Semi-formal, Random generators, Formal verification
Debug and comprehension: waveforms, debug systems, Behavior based systems which use formal technology
Infrastructure: Intelligent testbenches, Hardware Verification Langauages, Assertions

A good reference for verification is "writing Test benches by Janick Bergeron".

Model a buffer of 20 units in verilog

HI all, Which is the correct way of modeling a buffer of 20 units in verilog

Option 1 : #20 a = b ;
Option 2 : a = #20 b ;
Option 3 : #20 a <= b ;
Option 4 : a <= #20 b ;

Try to find the corner case in this verilog code

Hi Everyone, The following is a small verilog code which below models a flip-flop with asynchronous set/reset logic (active low). The model synthesizes correctly, but there is a corner case where simulation results are incorrect. What is the corner case?


always_ff @( posedge clk
or negedge rst_n // active-low reset
or negedge set_n // active-low set
)
if (!rst_n) // reset has priority over set
q_out <= '0; // reset all bits to zero
else if (!set_n)
q_out <= '1; // set all bits to one
else
q_out <= data_in; // d input assignment

Hope this question will tickle the technical senses of all the ASIC verification engineers