Weapons Procurement: Qualitative Requirements and Transparency in Evaluation

Mrinal Suman

Major General (Retd.) Mrinal Suman retired from the Indian Army in 2003 and currently heads the Defence Technical Assessment and Adivsory Service of Confederation of Indian Industry. read more

Procurement of new weaponry and equipment for the armed forces is a long, complex and arduous process. Funds involved are enormous and the quality of equipment selected has a profound influence on national defence potential. Therefore, most countries have evolved elaborate procedures aimed at procuring the most appropriate equipment at affordable prices.

Critical activities like determining parameters and evaluating competitive bids need to be carried out diligently while ensuring adequacy of equipment profile, financial probity, objectivity and transparency. Effectiveness of any functional procurement regime, inter-alia, depends on the following aspects:

• Formulation of cost-effective Qualitative Requirements (QRs) so that the users get what they need at minimum expenditure.
• Evaluation of technical and commercial proposals in a transparent, fair and well-defined manner by using scientific tools for objective inter-se comparison.

QRs are evolved to specify essential parameters of military equipment needed in a specified time period to counter a threat, fulfill other operational needs, or fill an equipment void. They spell out the users’ requirements in terms of functional characteristics in a comprehensive, structured and concrete manner.1 In other words, they define minimum performance attributes, corresponding to the task or tasks to be performed by the system.

As QRs form the basis of equipment philosophy, they are need based. They apprise the vendors about what is being sought and provide a wellset benchmark for subsequent inter-se appraisal of equipment tendered for evaluation by different vendors.

Since formulation of QRs is an important stage of the entire process, a highly deliberate and meticulous approach has to be adopted while determining them. QRs generally depend on the following factors:

• The nation’s operational doctrines and operational plans.
• The prospective enemy’s capabilities, his probable plans and tactics.
• The likely pattern of employment of the equipment in the obtaining terrain and climatic conditions.
• The current and anticipated technology levels.

Poorly conceived and imprecisely drafted QRs create confusion, lend themselves to misinterpretations, compromise quality of equipment, prove expensive and cause immense delays.2

Practices in Some Other Nations

The practice of asking the services to evolve QRs of equipment sought is a legacy of the Second World War. It is too rigid and does not cater for changing technology. Most of the developed countries have already adopted different procedures as per the level of technology mastered and the maturity of their indigenous defence industry.

The United States introduced Advanced Concept Technology Demonstration (ACTD) a decade ago. The concept aims at offering comparatively stabilised technologies to the defence forces and letting the commanders ascertain their suitability in the operational environment.3 Thus, it is left to the commanders to determine whether the equipment offered meets their requirement or further developmental work is required. In this methodology, it is not the military that demands development of new systems ab-initio. Advantage is taken of the nation’s technological prowess to tell the military as to what equipment can be made available. Thus, time taken to develop new technologies as per the military’s requirements is eliminated.

The ACTD concept can work best for countries which have a welldeveloped scientific base with multiple agencies working on different competing technologies. This approach has the added advantage of making the military aware of what is technically feasible in a given time frame, rather than seek equipment with over-ambitious and impractical QRs.

The Russians follow a bottoms-up approach in which initially only ‘baseline standards’ are evolved for a large variety of military equipment. These standards are grouped together to form ‘basic profiles’. These, in turn, help generate broad equipment contours with distinct characteristics. Profile of equipment, when translated into specific distinctive requirements, is called a ‘functional standard’. A functional standard is thus a document that lays down the parameters for the development of equipment. In other words, baseline standards are like building blocks, which are common to a large array of military systems. These are combined to get basic profiles of a range of equipment, whereby profiles get converted into functional standards to define a military product.4

Such an arrangement is ideal for a country which rarely imports any military hardware but develops its complete requirements indigenously. It is a highly cost- effective system as it exploits the technology mastered over a range of products. It reduces inventory and facilitates easy life-cycle support.

In Britain, the services are asked to provide basic Cardinal Points Specifications (CPS) only. These are operational parameters specifying performance requirements in very broad terms. This helps the Defence Procurement Organisation to study the projections in detail and decide on ‘make’ or ‘buy’ decisions, in consultation with the research/development agencies and the defence industry.5 Even procurement is carried out on the basis of CPS, which are made known to all the producers. This is a very ingenious method by which the producers, while conforming to CPS, can introduce innovative techniques and ideas. All products, which comply with the CPS, are trial-evaluated by the services to identify the most suitable ones for introduction into service. This also provides a common platform to judge different technologies for futuristic adaptation and further research. It is a highly practical methodology for a nation that produces as well as
imports military equipment.

Formulation of QR in India

After the inclusion of a projection in the acquisition plan, the sponsoring directorate is asked to finalise its QRs. All available books on the equipment and catalogues of the manufacturers are collected. The best characteristics of all known equipment are compiled as requirements. There is a general tendency to include as many features as possible to demonstrate the enormity and exhaustiveness of the work done.

Thereafter, the draft is circulated to various concerned agencies, other possible user directorates and maintenance directorate for obtaining their views/comments. Staff Equipment Policy Committee (SEPC), as constituted by the Services Headquarters, finally approves the QRs. In cases where commonality of equipment exists and standardisation of QRs is merited, a Joint Staff Equipment Policy Committee is constituted, with representatives of all the three services to formulate Joint Services Qualitative Requirements.

A review of the Indian system reveals the following weaknesses:

• QR formulation is a highly specialised task which calls for staff with flair, talent and a thorough knowledge of competing technologies. However, in several cases, Staff tasked to evolve QRs is not selected for any demonstrated competence. Thus, they remain untrained and ill-equipped for the task.
• There is a tendency to seek irrelevant, non-essential, unverifiable and unusable capabilities without reference to available stabilized technologies. No one questions the need of a parameter.
• There is a total lack of cost-consciousness. Cost, vis-à-vis minimum inescapable parameters, is never considered.
• Staff is unable to translate the required parameters into established universally accepted standards. This leads to multiple interpretations.
• QRs are spelt out in imprecise and indeterminate language.
• Services tend to make QRs ‘futuristic’ fearing rapid obsolescence during the protracted procurement drill. Moreover, as the life of any major military equipment is 15-20 years, users are wary of equipment becoming outdated during their long military life. Therefore, QRs generally take the shape of a well-compiled ‘wish list’ of utopian dimensions.
• At times, upgraded versions of existing equipment are sought on the basis of presentations by vendors.

Cost Efficiency Considerations

While pitching a parameter at a specific level, it has to be borne in mind that for every rise in level, there is an associated cost. Cost is a function of performance and the relationship between the two is not linear. As a matter of fact, cost increases in a geometric progression. Therefore, while fixing parameters, it is prudent to examine cost penalty since a minor acceptable moderation of a parameter may bring about huge savings. Every parameter must be judged for its inescapability.

There is need to resist the proclivity for demanding custom-made equipment as per own QRs in the very first instance since this is a costly option. Due to economies of scale, all efforts should be made to make do with equipment available off-the-shelf in the indigenous or world market. Similarly, all efforts should be made to utilise dual-use technology for defraying the cost of acquisition.

The Services should provide the range of performance parameters, with clearly specified minimum acceptable standards. Additional credit, through a system of multipliers, should be assigned for better performance. As it may not be possible to provide an acceptable range for all parameters, a well-considered mix of specifics and matrix is the best option.

Need for Separate QR for ‘Buy’ and ‘Make’ Cases

QRs for ‘buy’ and ‘make’ cases can never be the same. ‘Buy’ cases need contemporary and well-stabilised technologies which are widely available to generate sufficient competition. Equipment should be readily available for procurement and induction in the given time frame.

QRs for ‘make’ cases are based on futuristic technologies with uncertain time frames for their development and availability. QRs may also undergo periodic revision with changing milieu and acquisition of competitive technology/equipment by a potential adversary during the period of development. Till recently, common QRs were formulated both for ‘buy’ and ‘make’ cases in India.

Defence Procurement Procedure 2006, promulgated with effect from 01 September 2006, has prescribed a separate procedure for the formulation of QRs for ‘make’ cases pertaining to high technology complex systems. The sponsoring Service Headquarters is required to evolve broad based and realistic Preliminary Services Qualitative Requirements (PSQR). PSQR contain essential parameters (key performance attributes) and desirable parameters (for later development).6

Essential parameters need to be based on the proven state-of-the-art technology available in India or abroad. On the other hand, desirable parameters are based on futuristic/emerging technologies. PSQR would be subject to review in consultation with the Service Headquarters as the development progresses. Any amendment in essential parameters after the preliminary design phase would need the sanction of Defence Acquisition Council (DAC).

Introduction of the above provisions fails to address the central problem of determining QRs at the proposal stage itself. Additionally, as the ‘make’ procedure restricts itself to proven and matured technologies available in India or abroad, it really makes it akin to the ‘buy’ process, albeit limited to Indian vendors. In case the required technology is available in India, minimal effort is needed to put together the system indigenously. However, if the technology has to be imported, indigenous defence industry will need to resort to co-production or outright purchase of technology along with necessary equipment. There is no research or development content in both the cases.

The new dispensation has considerably complicated matter. India should follow a simple well-defined procedure without getting mired in quibbling and pedantry. Service Headquarters should initially devise purely military characteristics, which pertain primarily to the functions to be performed by equipment, either alone or in conjunction with other equipment in use. These Operational Characteristics should define performance parameters in broad terms only. Thereafter, DAC should deliberate upon the case to decide on whether to adopt a ‘make’ or a ‘buy’ approach.

Once a ‘buy’ or ‘make’ decision is made, the case should be returned to the Service Headquarters to frame Qualitative Performance Requirements for Purchase (QPRP) for ‘buy’ cases and Qualitative Performance Requirements for Development (QPRD) for ‘make’ cases respectively. QPRP are based on the equipment currently available in the world market, whereas QPRD
are based on futuristic technologies under development.

In ‘buy’ cases, QPRP should be forwarded to the Acquisition Wing, where a Technical Parameters Committee (TPC) is constituted under the concerned Technical Manager to generate Technical Characteristics (TC). TPC should have representatives of Director General Research and Development (DRDO), Director General Quality Assurance (DGQA), maintenance directorate and the sponsoring service amongst other concerned agencies. Experts from academic institutes and the private sector should also be co-opted for wide ranging consultations.

However, in ‘make’ cases, the Service Headquarters should forward QPRD to DRDO. DRDO needs to have a standing Research Oversight Committee (ROC) to analyse all QPRD and perform the functions of a think tank, where various technological alternatives are brain-stormed in depth. ROC should be tasked to produce Qualitative Research Requirements (QRR) for the equipment to be developed. QRR is primarily a technical road map, which broadly spells out the technology to be adopted, assignment of responsibilities and outline contours of various phases of development with time frames. Once the Defence Development Board, under the Scientific Advisor to the Defence Minister, approves the QPR, it acquires the shape of a Policy Directive and acts as the basic document for the development of that equipment.

Thereafter, the case is processed by DRDO, who define Technical Specifications (TS) of the equipment (which flow from QPR). These specifications relate to actual design development, production processes and engineering. Close interaction and periodic joint reviews with the user
service are maintained throughout the development phase. A schematic representation of the whole decision-making process to decide ‘buy’ and ‘make’ cases is given at Fig. 1.
Fig-1: Schematic Representation of ‘Buy’ and ‘Make’ Cases

The proposed mechanism has the following major advantages:

• It ensures separation of formulation of parameters for immediate procurement and development.
• Service Headquarters prepares realistic and well spelt out Qualitative Performance Requirements after obtaining extensive inputs at DAC meeting, where other services and experts are also present.
• Technical Characteristics are not left to the military but are formulated by a technical committee after wide-ranging discussions, thereby ensuring their applicability and practicality.
• Formulation of Qualitative Research Requirements is assigned to the scientific community. Close and regular joint monitoring of the development process is thus institutionalised.

Need for a Culture of Cost Consciousness

Since the services are the ultimate users, they should have the final say on the issue of parameters sought. However, it is equally important that the services take informed decisions. They ought to be aware of the financial impact of parameters fixed by them. They have to be apprised that the
relationship between performance and cost is neither linear nor in direct proportion since defence equipment is technology-sensitive. Thereafter, it should be left to the Services to either stick to the previous parameters or moderate them to obtain better ‘value for money’.

Use of equipment which is commonly available off the shelf in the civil market is a highly cost-effective choice. The services must explore this option at the outset. Similarly, an endeavour should be made to seek dual-use technology for economies of scale and up gradation of the industrial base in the country. Instead of seeking fully developed equipment at the outset, it is prudent to progressively induct improved equipment as they are developed. The British system of Mark I, Mark II and Mark III or the US Block system should be adopted.

In the new Defence Procurement Procedure, there is no provision for the grant of deviations after the issuance of Request for Proposals (RFP). The whole case has to be aborted and initiated afresh if none or only one vendor meets all the parameters. It is a highly restrictive stipulation. It will force the services to pitch their parameters at base levels to ensure adequate response, thereby depriving the services of equipment with higher performance characteristics.

It would be ideal for the services to formulate their QRs in the form of a bracket with lower and upper limits specified. The lower limit should provide the minimum benchmark that the particular equipment must meet. Extra credit could be given to the equipment for attaining higher performance standard, albeit within the specified range with the upper cap. Relative weightage has to be assigned to different parameters for the purpose. Such an arrangement will result in the development of a matrix of all quantifiable parameters. A well-considered mix of specifics and matrix is ideal for India.

It has also to be appreciated that cost efficiency does not mean identifying and bargaining with L1 for a given set of QRs alone. It is, perhaps, more important to obtain value-for-money by considering QRs with respect to cost and moderating them, if operationally feasible. This can only be done by applying performance-cost analysis techniques after detailed technology scan and market survey. Minor acceptable moderation of QRs may affect huge savings. It must be reiterated that the services must continue to have the final say on performance-cost trade off.

Weighted Aggregate Performance Index (WAPI) is worked out by assigning inter-se weightage to various performance parameters as per their criticality to operational exploitation of the equipment. Such an index helps in determining the type and level of technology needed, and its impact on cost.

As can be seen in Fig. 2, Equipment ‘A’ scoring 220 on WAPI scale will cost a great deal more than Equipment ‘B’ achieving 210 on WAPI scale due to the technology jump involved. Additionally, Equipment ‘A’ will have yet unstabilised technology with very few producers. On the other hand Equipment ‘B’ will have the benefit of matured, tested and inproduction technology with multiple producers. Thus, WAPI-Cost analysis acts as a scientific tool to help identify the most cost-effective acceptable option.

Transparent Evaluation Criteria

Once proposals from the invited vendors are received, they have to be examined for their comprehensiveness and compliance with the stipulated requirements. It entails detailed examination of all aspects to eliminate subsequent mis-interpretations. It has also to be ensured that all vendors understand the laid-down stipulations and submit their responses in an unambiguous manner.

Proposals are submitted in two separate envelopes (technical and commercial), albeit simultaneously. Technical proposals are opened to identify vendors whose products meet all QRs and can be procured. This process is called technical evaluation. Subsequently, commercial bids of technically successful vendors are opened for comparative price evaluation.

Technical Evaluation

Evaluation of technical proposals is a three step process: (i) paper evaluation of proposals to shortlist vendors for trials, (ii) field evaluation to validate performance claims and (iii) General Staff evaluation to identify equipment to be recommended for induction into service.

Paper Evaluation

Paper evaluation of proposals is carried out by a Technical Evaluation Committee (TEC) constituted under the Service Headquarters. It is a broadbased committee with wide representation. In case technology transfer is sought, nominated production agency is also co-opted. It scrutinises all technical proposals for their QR compliance. The committee seeks clarifications from vendors, if required. As stated earlier, if none or a single vendor is found to be complying with all QRs, the case has to be retracted and fresh QRs formulated.

A number of difficulties are faced in carrying out effective paper evaluation. These include the following:

• Not all members of TEC possess the required knowledge to scrutinise technical proposals of newer technologies.
• Many vendors forward standard product brochures with complete disregard to QRs. TEC cannot ignore their offers and has to scan small print to ascertain compliance.
• Many experienced vendors make false claims hoping that required deviations will be granted subsequently. Honest vendors, who state facts, may get rejected for non-compliance.
• Some vendors claim compliance only to have their equipment field tested in varying terrain and climatic conditions for their further development at India’s cost.

Field Evaluation

It is carried out under the aegis of Service Headquarters. The overriding aim is to confirm compliance of parameters in actual terrain and climatic conditions. Maintainability aspects are also examined. It is normally carried out on ‘No Cost No Commitment’ basis. Service Headquarters issues the trial directive which the trial unit has to adhere to.

Some of the common problems faced are:

• Trial of new equipment with differing technologies is a highly specialised task. Trial units are totally untrained and ill-equipped for the same
• Trial directives are issued as a matter of routine
• Many aspects are indeterminate and are viewed differently by trial units.
• Trial reports tend to be subjective as per the views of commanders in the chain and couched in generalities.
• New parameters are incorporated by field commanders while giving final recommendations.