Pillar II: Excellence in Measuring Patent Quality

American Intellectual Property Law Association (AIPLA)

The American Intellectual Property Law Association (AIPLA) comments on all three of the PTO Pillars and the six proposals thereunder. The AIPLA suggests certain modifications that would make the process for requesting prosecution reviews more effective, such as publicizing how and when review applications are selected from the larger pool. Further, the AIPLA advocates certain changes to the automated pre-examination search, such as expanding the search algorithms of the Patent Linguistic Utility Service (PLUS) tool, considering corresponding applications in Foreign Offices, and allowing public access to search results for a fee. As for improving excellence in measuring patent quality, the AIPLA is concerned that the current metric is more directed towards improving production than quality, and provides alternative performance metrics that should be measured. Lastly, the AIPLA applauds the PTO’s efforts to increase access to in-person interviews with examiners. The AIPLA sees this as the most effective way of increasing both excellence in patent quality and improving communication between examiners and applicants. The comment also proposes other changes to existing procedure to improve efficiency and effectiveness of the examination process.

Click here to view

Chartered Institute of Patent Attorneys (CIPA)

This UK-based attorney organization is guardedly supportive of the Pillar 1 proposals for applicant requests for review and for greater usage of automated searches of prior art. It warns that the side effects of enhancing the clarity of prosecution records by making them more detailed could backfire by making prosecution more complex and and foreclosing accurate judicial claim construction; in the alternative, it observes that PTO records could be made significantly more transparent simply by making its databases more integrated and user-friendly. It criticizes of the PTO’s numbers-driven approach to quality, stating that Pillars 2 and 3 cannot be achieved unless examiners and reviewers are given more time to do their jobs diligently. Beyond the Pillars, it notes that quality could be enhanced by making better use of the international Common Citation Document and by broadening third parties’ opportunities to comment on pending applications.

Click here to view

Coalition for Patent and Trademark Information Dissemination (CPTID)

The Coalition for Patent and Trademark Information Dissemination (CPTIP) comments on Proposal 2 (Automated Pre-Examination Search) and Proposal 3 (Clarity of the Record) under Pillar I and Proposal 4 (Review of and Improvement to Quality Metrics) under Pillar II. The CPTIP does not support the PTO’s proposal to provide examiners with automatic pre-examination search tools, fearing that over-reliance on an automatic search will decrease patent quality because it will discourage examiners from conducting their own detailed manual searches. If the proposal is to be implemented, the CPTIP recommends using private sector tools, ensuring data quality, and public access. For Proposals 3 and 4, the CPTIP suggests enhancing quality checks of applicant inputs, maintaining the publication process, using tools to quantify variations in examiner performance, correcting published patent text, enriching metadata through enhanced content tagging, and retaining consistency of published content.

Click here to view

Electronic Frontier Foundation (EFF);
Engine Advocacy;
Public Knowledge

The Electronic Frontier Foundation (EFF), Engine Advocacy, and Public Knowledge propose an additional pillar in the PTO proposals, called “”Excellence to the Public.”” First, the EFF suggests that treating patent applicants as “”customers”” and promoting better “”customer service”” fails to reflect the true public service role of patent examiners. Low quality, overbroad patents pose a threat to innovation through the legal costs required to invalidate illegitimate claims. The EFF advocates defining patents in public interest terms and applying more stringent patent requirements. In relation to Proposal 1 under Pillar 1, the EFF suggests there should be a greater focus on reducing the error rate regarding allowances. For Proposal 2, the EFF addresses concerns such as over-reliance on and manipulation of automated search tools. The EFF supports Proposal 3, asserting that explicit claim construction, recordation of examiner interviews, and statements outlining the reasons for allowance should be adopted. For Proposal 4 under Pillar 2, the EFF proposes shifting the focus of patent quality metrics from process quality to output quality, and proposes two new metrics. In relation to Proposal 5 under Pillar 3, the EFF proposes greater flexibility and increased time in the compact prosecution model. Lastly, for Proposal 6, the EFF advocates expansion of examiner interviews, so long as these interviews are properly documented and all relevant presentations are filed.

Click here to view

Intellectual Property Owners Association (IPO)

The IPO addresses all six of the PTO’s proposals. Proposal 1 under Pillar 1 (prosecution review of selected applications) – This could be implemented via a combination of formal and informal procedures for requesting review. Proposal 2 under Pillar 1 (automated pre-examination search) – For these searches, examiners should have access to a modern search interface and a wide variety of prior art. The public should have access to the same search tool, so that applicants can identify prior art before filing applications. Proposal 3 under Pillar 1 (clarity of the record) – Clarity is a laudable goal, and interactions such as interviews should be recorded more carefully. But the IPO urges caution, because unreasonable expectations of claim specificity could impose too great a burden on applicants. Proposal 4 under Pillar 2 (quality metric) – The IPO offers several specific suggestions for tweaking the metric. It also uses the PTO to use quality metrics to gauge the effectiveness of examiner training. Proposal 5 under Pillar 3 (review of the prosecution model) – Final rejections create an unnecessary “artificial pause” in the prosecution process and should be eliminated. A second non-final office action is a good idea, but should be available without an additional fee.  Proposal 6 under Pillar 3 (interviews) – Enhanced availability of in-person interviews would be helpful because effective communication sometimes requires in-person contact.

Click here to view

Internet Association

The Internet Association is particularly enthusiastic about the PTO’s proposal regarding clarifying examination records (Proposal 3 under Pillar 1) and views this as a mechanism to identify and appropriately limit functional claiming in software patent applications. It also supports the PTO’s plans to automate pre-examination searches (Proposal 2 under Pillar 1), revise patent quality metrics (Proposal 4 under Pillar 2), and review the current prosecution model (Proposal 5 under Pillar 3), but argues that the PTO should do even more than proposed on these fronts. However, it is opposed to the proposal to enable mid-examination review (Proposal 1 under Pillar 1).

Click here to view

New York Intellectual Property Law Association (NYIPLA)

The NYIPLA proposes that third parties be allowed to make more detailed comments when submitting prior art relating to a pending application. It also offers feedback on each one of the PTO’s proposals: Proposal 1 under Pillar 1 (prosecution review of selected applications) – Review of applications by the Office of Patent Quality Assurance (OPQA) department would be beneficial, but the OPQA is too thinly staffed to make much of an impact. The NYIPLA is curious how the OPQA would select which applications to review. Proposal 2 under Pillar 1 (automated pre-examination search) – Automated searches could have benefits, but the PTO must be careful that examiners do not rely on such searches to the exclusion of making a more thorough manual search. Proposal 3 under Pillar 1 (clarity of the record) – It would be unduly burdensome to explicitly define every claim to the level of detail envisioned in the PTO’s proposal. Interview summaries need not be as detailed as the PTO proposes. Nonetheless, the records generated at the end of an examination should be more detailed than they are currently. Proposal 4 under Pillar 2 (quality metric) – The composite metric may be beneficial for internal USPTO use, but it is confusing for practitioners. Proposal 5 under Pillar 3 (review of the prosecution model) – A second non-final office action is a good idea, but should be available without an additional fee. Proposal 6 under Pillar 3 (interviews) – More interviews are good, but due to expense and logistical concerns, in-person interviews are not always feasible. Telephone and videoconference interviews should also be available.

Click here to view

Software & Information Industry Association (SIIA)

Software & Information Industry Association (SIIA) believes that the issuance of high quality patents is critical to an effective patent system and should be the primary goal of the PTO. To this end, SIIA suggests that applicants be required to clearly define the scope of their patent claims, as well as identify and define key claim terms. It recommends that the PTO improve the frequency and focus of the training programs presently offered to examiners. The PTO should digitize prior art and make it publicly available, and continue to provide examiners with manual search tools. To improve quality metrics, the PTO should assess rejections and overcoming of rejections in each art unit. SIIA also recommends using advanced text analytic tools to identify applicant-generated errors, which would be aided by the technologies of commercial providers like LexisNexis and Thompson Reuters. SIIA believes that quality consideration should prevail over compact prosecution. Finally, SIIA believes that video interviews are an acceptable alternative to in-person interviews.

Click here to view

U.S. Section of FICPI (International Federation of Intellectual Property Attorneys)

This comment addresses all of the PTO proposals. First, the comment suggests that prosecution review by the Office of Patent Quality Assurance should be limited to applicants or practitioners with standing in the application; third parties should not be allowed to participate in this program. Second, the comment supports an automated pre-examination search. Third, the comment finds that a prosecution record that includes a more extensive and legally correct claim construction analysis by the examiner would be beneficial to applicants and the public; however, recording of examiner interviews is discouraged. Fourth, the comment recommends improving the Quality Composite Metric and finding ways to enhance examiner training. Fifth, it suggests that the finality of the second Office Action severely limits the applicant’s ability to reply to the examiner’s response to amended claims, and necessitates RCEs; it suggests providing an option for applicants to convert the final Office Action into a nonfinal Action, perhaps for a fee. Lastly, the comment finds that applicants would benefit from a more flexible system of in-person interviews that allows for more interaction with the examiner.

Click here to view

Beem Patent Law Firm

This law firm cautions that erroneous rejections, not just grants, are a serious problem in the patent system. Poor quality rejections are costly both for the PTO and for applicants who cannot afford to continue prosecuting rejected applications. Every patentable claim should be allowed as expeditiously as possible; to that end, examiners should put more effort into identifying patentable subject matter and working with applicants and attorneys to agree on allowable claims.

Click here to view

Askeladden L.L.C.

Askeladden proposes several extensions of the PTO’s proposals: (A) improve the search tools used by examiners by expanding and consolidating the library of prior art into a single interface, allowing the public to upload new literature to the database, and using modern search algorithms; (B) continue to enhance examiner education; (B) give examiners more time and count system credit to review prior art following an unusually large or untimely information disclosure statement; (C) allow third parties to make preissuance prior art submissions and give examiners the time and count credit to review these submissions; and (D) track quality metrics over time to assess the success of implemented initiatives. Askeladden also notes its approval of increasing clarity of the record (Proposal 3 under Pillar 1), observing that such clarity will give applicants the chance to timely remedy misunderstandings and will inform future litigation.

Click here to view

Cisco Systems, Inc.;
Google Inc.

Under Pillar 1, Google and Cisco suggest that the PTO revise the count system, focus performance appraisals on quality, and monitor granting of signatory authority. The PTO should strictly enforce Section 112 by requiring applicants to define key claim terms, enforcing the enablement and written description requirements, and by combating gamesmanship in continuation applications. Further, application of Section 103 should be improved through increased obviousness training and use of PTAB proceedings as guidance. PTO should also require applicants to fully distinguish the prior art while itself providing more clarity on the art of record and consistent evaluation of preissuance submissions. Proposal 1 would be an inefficient use of resources, whose goals may be better realized through an expansion of the Ombudsman Program. An automated pre-examination search could effectively supplement the examiner’s full search, and utilize Google Prior Art Finder. Under Pillar 2, the commenters suggest that the PTO provide total data transparency by coding all examination activity and using this data to perform a quality audit. Under Pillar 3, the commenters suggest that the PTO provide negotiations training to patent examiners, replace final rejections and RCEs with a limited number of multiple Office actions, and record interviews.

Click here to view

Dell Inc.

Emphasizing that it is a frequent target of trolls, Dell is enthusiastic about all three Pillars. It particularly supports using search automation to improve efficiency (Pillar 1), collecting more detailed “art area”-level data on examination processes (Pillar 2), and giving applicants more opportunities to interact directly with examiners (Pillar 3).

Click here to view

IBM Corporation

IBM’s comment addresses all six PTO proposals. With regard to work product, IBM proposes opening the Office of Patent Quality Assurance (OPQA) to review Section 101 rejections to ensure well-reasoned rejections. It also proposes expanding the pre-examination search to include non-patent references, alongside analytics to narrow the relevance of the results. Third, IBM supports making explicit claim construction early on in the prosecution, and ensuring informal conversations do not become crucial parts of the record, to avoid accidentally narrowing claims or getting adverse construction. Lastly, IBM suggests clarifying reasons for allowances. With regard to quality, IBM proposes using clear and publicly available metrics of quality, and giving public access to data points such as grounds for rejection and inclusion of data points on reasons for rejections. With regard to customer service, the PTO should limit the available number of non-final office actions or increase the fee after the second non-final office action to encourage quicker resolution of patent issues. Lastly, the PTO should provide more detail on the capacity of examiners to conduct interviews.

Click here to view

Innovation Alliance

The Innovation Alliance (IA) is a coalition of R&D focused companies; its comments address each of the six PTO proposals in turn. IA believes Proposal 1 has merit, and urges consideration on whether applicants should identify more specific issues forming the basis of the review request. IA also raises concerns with making the review part of the public record. For Proposal 2, IA supports improving the search functionality of examiners, but set out some technical criteria for selecting such an automation system. As for Proposal 3, IA opposes mandating explicit claim construction in the record because it could potentially decrease clarity and increase costs. In its comment, it details significant issues with this portion of the proposal. IA also opposes recording examiner interviews, believing that both examiners and applicants would be reluctant to have a complete record. With Proposal 4, IA supports improving quality metrics, as long as they accurately reflect work product quality. In relation to Proposal 5, IA supports allowing applicants to pay for an additional Office Action, but urges fair consideration of a broad range of proposed concepts. Lastly, for Proposal 6, IA believes allowing off-site interviews with examiners merits consideration, but is concerned with procedural aspects such as confidentiality, examiner safety, and the costs and inconveniences of travel.

Click here to view

Minesoft

This comment addresses Proposal 2 under Pillar 1 (Automated Pre-Examination Search). It recommends that USPTO examiners should use an automated pre-examination search tool called PatBase, a global patent family database. The tool includes access to patent family data, bibliographic information, and the full text of patent documents from the major patenting authorities. This will enhance the examiner’s ability to conduct an efficient, relevant, and comprehensive prior art search using a single resource. Moreover, daily electronic feeds into PatBase will provide examiners with current and up-to-date coverage. The comment further proposes using a tool that has original language search, translation, search customization, integrated advance analytics, and automatic alerting service capabilities.

Click here to view

Patent Navigation Inc.

The PTO should give procedural priority to applicants who provide documentation of claim support and an antecedent basis for claim terms. This would incentivize applicants to conduct available quality control assessments and avoid problems arising under Section 112. The result would be greater certainty in claim scope and economization of PTO resources.

Click here to view

Cohen, Lauren

This academic researcher (Note: a colleague of Umit Gurun, who also commented separately) notes that her research has revealed empirical differences between the patent assertion behaviors of non-practicing entities (NPEs) and those of practicing entities (PEs). NPEs’ patent litigation/assertion profiles tend to indicate trolling rather than protection of legitimate commercial interests. The commenter thus recommends that the PTO use these markers to identify potentially low-quality patents that may warrant further investigation.

Click here to view

Evans, Daniel

This comment notes the importance of a strong, clear patent system in providing market incentives for research and development, and highlights the lack of attention paid to each individual patent. The comment suggests that third party prior art submissions and/or software tools for sorting through prior art should be used to increase the attention paid to each application, and thus provide more predictability for the market.

Click here to view

Frakes, Michael;
Wasserman, Melissa

Based on their study, these authors discovered that the examiners’ time allocations to review patent applications are causing patent examiners to allow low quality patents. As examination time is cut roughly in half, grant rates rise by as much as 9 to 19 percentage points. Currently, patent examiners receive less examination time as they get promoted. The authors suggest that current scaling of the time allotments upon promotion is too aggressive. They suggest that examiners who are promoted face new time allocations that are less than a 10 to 15 percent decrease from their previous quotas.

Click here to view

Franklin, Thomas

This comment addresses Proposal 4 under Pillar 2, suggesting that quality metrics should focus on substantive quality, not production results. The comment finds that quality metrics should consider not only the quality of allowances, but also the quality of office actions (OAs) and other interactions with the applicant, including the quality and clarity of written rejections and quality and relevance of cited references. The author also recommends producing statistics for examiner art units, making available such data internally, seeking applicant feedback on examiners, tracking appeal decisions in quality metrics, allowing off-site work only to top examiners, and making supervisors accountable for examination quality.

Click here to view

Grantham, Robert

This comment stresses the importance of thorough, accurate prior art research. Regarding Pillar 2, the current Quality Composite Metric includes no assessment of the quality of the prior art research done both by the applicant and the PTO. At a more fundamental level, the PTO should consider whether and why its examiners’ search practices differ qualitatively from the private sector’s. Regarding Pillar 1, pre-examination search is an excellent idea, but the automation of such searches should not be taken too far: dispensing with the “”human element”” of prior art research would be a mistake.

Click here to view

Guerrini, Christi

This researcher proposes adopting a “validity-plus” definition of patent quality that includes (1) an assessment of the patent’s likely validity at the time of prosecution and (2) the comprehensibility of the patent’s scope and terms, but excludes (1) the patent’s commercial and social value, (2) the patented invention’s commercial and social value, (3) the PTO’s operational efficiency, and (4) user satisfaction with the PTO’s services.

Click here to view

Gurun, Umit

This academic researcher (Note: a colleague of Lauren Cohen, who also commented separately) hypothesizes that the patents typically held by non-practicing entities (NPEs) probably tend to be different, and perhaps of lower quality, than practicing entities’ (PEs) patents. For a clearer understanding of how NPEs’ patents compare to PEs’, a large-sample empirical analysis is needed. The PTO could support such research by making available data that allows researchers to assess patents’ quality in “some summary sense.”

Click here to view

Katznelson, Ron

This comment notes that there is a fundamental asymmetry in the observable impact of examination errors, and thus the relative costs of rejection errors compared to allowance errors have been largely ignored. Thus, the comment suggests that the USPTO should create a balanced examiner incentive system that includes rejection error measures. To do so, it is suggested that the USPTO should conduct statistical performance studies and measurements in order to design a better examiner production-goal system. Moreover, the USPTO should align the allotment of resources with examination burdens required to achieve acceptable examination error rates. The comment also proposes a method for improving examination quality by adjusting the time allotted for examination based on objective quality criteria. Employing various economic models, the comment ultimately proposes a modification to the examiner count system.

Click here to view

Katznelson, Ron

This comment suggests various reforms to the USPTO’s operations. First, it suggests strengthening the examiner force by providing greater financial resources for competitive pay, modifying examiner incentives, and allowing for continuing professional development. Next, the comment suggests that there should be greater investment in a patent classification system. The comment also notes that, in looking at quality, there is an inherent bias towards observable allowance errors as opposed to rejection errors, and the focus should be on examination quality and not just (granted) patent quality. The comment provides some statistical evidence to support its claims. Lastly, the comment proposes changing the USPTO’s operational metrics, adopting a more balanced examiner incentive system that takes into account rejection errors, conducting statistical performance studies to design a better examiner production-goal system, and adopting deferred examination procedures.

Click here to view

Keller, Martin

This comment addresses the importance and need for pre-application searches of prior art references. The comment also briefly addresses Proposal 2 under Pillar 1 (Automated Pre-Examination Search).

Click here to view

Lemley, Mark

This commenter suggests that the PTO should take several steps towards identifying and appropriately limiting functional claiming in software during the examination process, for instance, by requiring the applicant to clarify their claims on record.

Click here to view

Ourednik, John

Ourednik claims that any quick and easy means to measure patent quality will not actually measure a patent’s quality, as it will not indicate the merits of the rejections or the novelty claimed by the patent. To truly measure patent quality, each patent needs to go through a subsequent review to ensure that patents represent novel and non-obvious inventions, although this would require PTO to divert more of its time and resources.

Click here to view

Stein, David

This comment highlights four high-level issues relating to the PTO’s definition of patent quality, such as its under-emphasis on incorrect application rejections, as well as subjective patent quality evaluation. The comment makes several detailed recommendations, such as including a “persuasive” quality metric, and analyzing metadata on examiners’ production counts. In regard to the latter, the comment provides sample summary metrics that could assist examiners in improving the quality of their office actions.

Click here to view

Wolff, John

This comment focuses on present, unresolved issues in the patent application system and potential issues with the Pillars proposed by the PTO. In addressing Pillar 2, the comment cautions the implementation of throughput and efficiency metrics as a measure of quality, as they may ultimately lead to negative effects on quality. Further, the comment discusses the present issue of practitioners who draft overly broad claims, which may attract litigation that inhibits their commercial success. The comment further proposes several additions to the Patent Quality initiative, such as partnering with educational institutions to educate inventors about basic patent application skills, creating a simplified patent manual to assist inventors who wish to apply for patents pro se, providing clarification for ambiguous MPEP rules, and modifying MPEP sections 101, 102, and 103 in order to limit patents granted for “trivial” improvements.

Click here to view