People ask us a version of the same question all the time. How good is PQAI, really?

It is a fair question, and a hard one to answer honestly. PQAI is well-known for novelty searching and is popular among inventors. If you have an invention idea, you can describe it in plain English, and PQAI’s semantic search can surface similar inventions and research papers that already exist, helping you decide how to proceed with your idea. 

But that is not the only place where prior art search matters. 

The search gets considerably harder once a patent has already been granted and somebody is investigating its validity. In an invalidity search, IPR preparation, or litigation-related prior art research, finding patents that are broadly similar is not enough.

You may be looking for very specific earlier disclosures that connect to particular claim limitations. So we wanted to put PQAI to a harder test.

Rather than deciding ourselves whether the results looked relevant, we found a case where expert searchers had already identified prior art against an asserted patent and the winning references had been publicly disclosed.

That gave us an independent benchmark against which we could test PQAI and more clearly answer the question: how good is PQAI, really?

Here is the test we ran.

We Used an Independently Judged Prior Art Challenge as the Benchmark

Did you know that Unified Patents runs PATROLL, a programme where cash bounties are awarded to people who find strong prior art against patents being asserted in litigation? These contests are open to the public. Submissions are reviewed, and the winning references are publicly announced.

For one of its recent challenges, Unified Patents worked with the Cloud Native Computing Foundation through the Cloud Native Heroes Challenge.

The challenge focused on US8379538B2, owned by Valtrus Innovations Ltd.

Several prior art submissions were made against the patent, and Unified Patents later announced the winning references.  

Source- Unified Patents

They were:

• US6122664
• US7454496
• RFC 2819, Remote Network Monitoring Management Information Base
• DMTF DSP0141, CIM Metrics Model White Paper

That gave us a useful external benchmark because these references had already been identified and publicly disclosed before we ran our PQAI test.

Our goal was now to see whether PQAI could independently surface the same prior art.

Putting US8379538B2 To The Test 

To conduct the test, we copied Claim 1 of the US patent into the PQAI search interface and added the priority date of June 22, 2005. We wanted to make this a demanding test, so rather than describing the idea behind the patent in plain English, we used the actual claim language.

Source – PQAI

We did nothing else. There was no Boolean query, keyword extraction, or search by assignee. We copied Claim 1 exactly as granted and entered it into PQAI. That was important because we wanted to test what semantic search could surface from the claim itself.

Moreover, this is also closer to how PQAI can complement a traditional prior art search workflow. The priority date also matters because a prior art search is not simply a search for technically similar patents. The timing of a reference matters as well.

Without the relevant date constraint, you can easily surface later patents that describe the same technology but do not help with the prior art question you are investigating.

Once the claim and date were entered, we ran the search.

PQAI Surfaced Prior Art From Several Jurisdictions

The first thing we noticed was how broad the result set was. PQAI surfaced 100 results, and they were not limited to US patents alone. There were  Japanese, Korean, European, Chinese, WIPO, German, Brazilian, British, French, Mexican, Taiwanese, and Canadian filings too.

Below, we have added a screenshot of the exported dataset, which is also available in Excel format.

This breadth of results is useful in prior art research, as you never know whether a technically relevant disclosure may exist in a Japanese, European, Korean, or international filing.

FYI, the first result was the published application associated with the patent we were testing. That is obviously not prior art against itself. But it gave us a useful sanity check.

That showed us that the query was representing the technical subject correctly before we started looking deeper into the results.

For an actual invalidity analysis, the patent itself and relevant family members would of course need to be excluded from the candidate prior art set. But for this benchmark, the self-match gave us a useful starting point.

From there, we continued going through the remaining results.

PQAI Also Found Closely Related Patents That Were Not The Winning References

Before we reached either bounty-winning patent references, we found several results with meaningful technical overlap. One good example was US5729472A. The patent is titled “Monitoring architecture” and was published in 1998. It describes a system with a central point of control and multiple managed systems containing monitoring agents.

Those monitoring agents can include watchdog modules, rule evaluators, and action modules. Monitoring policies can be evaluated locally rather than requiring everything to be processed centrally.

The connection to the ‘538 patent is easy to see. Both describe an architecture in which behaviour across the monitoring environment can be determined by information defined elsewhere in the system.

But US5729472 was not one of the winning Cloud Native Heroes submissions.

You see, Semantic relevance can bring a document into the review set. However, it still takes a researcher or patent professional to determine whether the reference actually addresses the required claim limitations and how useful it is for the search.

This is what a real prior art result set looks like. You get documents with different levels and types of technical overlap, and you investigate them.

The benchmark was whether the independently identified winning references were somewhere inside that set.

The first winning patent appeared at rank 71

At rank 71, we found the first target: US6122664

The patent is titled “Process for monitoring a plurality of object types of a plurality of nodes from a management node in a data processing system by distributing configured agents.”

The patent describes a monitoring system where configuration is established at a management node and distributed to autonomous agents located on the nodes being monitored. While the terminology is different from the ‘538 patent, the technical connection is still visible.

The later patent talks about a machine-readable monitoring model and monitoring elements adapting their operation based on the configuration defined by that model. US6122664 approaches the system differently, but it similarly connects centrally defined monitoring configuration with the behaviour of distributed monitoring agents.

PQAI’s claim mapping also helps show why the patent surfaced. It breaks the searched claim into individual elements and points to passages in US6122664 that it considers relevant to each one. In this case, the mapping connects the claim’s monitoring-model and configuration requirements with passages discussing management configuration, monitoring modules, and agents operating across monitored nodes. This was the first of the two US patent references that had independently won the Cloud Native Heroes bounty.

The Second Winning Patent Appeared At Rank 90

By the time we reached the ninth page of results, we found US7454496.

Filed by IBM in 2003, the patent is titled “Method for monitoring data resources of a data processing network.”

This reference describes monitoring requirements that can specify what resources should be monitored, what data should be collected, how frequently it should be delivered, and other characteristics of the monitoring process.

Again, the language differs from the ‘538 claim. But there is an important semantic relationship.The monitoring process is driven by a structured description of what needs to be monitored and how monitoring should operate.

PQAI’s mapping makes the basis for that match easier to see. For the first part of the claim, it points to passages discussing the “mode of monitoring data collection,” “derived metrics,” monitoring agents, and requirement descriptions associated with monitored resources.

In the second part, where the ‘538 claim requires an element to read the monitoring model, PQAI highlights passages around a computation model for derived metrics, requirement descriptions, a “reading and parsing mechanism,” and stored instructions. In other words, PQAI is connecting the claim’s idea of configuration-driven monitoring with passages in the winning patent describing how monitoring requirements and instructions influence what the system measures and processes.

The mapping does not by itself establish that every limitation of the claim is disclosed by US7454496. That still requires a patent professional to read the reference closely. What it does show is why PQAI considered this patent technically relevant despite the two patents using different terminology.

That is the kind of relationship a pure keyword search cannot easily surface when two patents describe related technical concepts using different language. And with US7454496 found, both US patent references from the published Cloud Native Heroes challenge results had now appeared in the same 100-result PQAI search.

The Ranks Tell Us Something Important About How PQAI Should Be Used

There is one thing worth mentioning before we conclude. If someone had checked only the top 20 or 40 results, they might assume invalidity search is not a strong use case for PQAI. But that would miss an important part of what the result set shows.

If you go through the results, you can see that at rank 6, PQAI surfaced US20080275985A1, which is closely related to the second winning patent, US7454496.

In fact, when we clicked on More like this beneath the US application on the PQAI interface, a set of 99 results appeared, among which US7454496 surfaced at the top. 

So although the exact winning reference appeared later at rank 90, PQAI had already identified closely connected subject matter much earlier in the result set.

This is useful because prior art searching does not always follow a straight path from query to final reference. Patent families, continuations, divisionals, related applications, and similar disclosures can give a researcher useful paths to follow even when the exact document they ultimately rely on appears further down the list. 

Before We Go

For years, PQAI has been naturally associated with novelty searching. However, that’s not where it ends. Invalidity searches are another useful use case, especially because traditional patent searching can leave analysts working through hundreds or even thousands of results across different keyword, classification, and citation searches.

PQAI helps make that process easier. You can enter the claim of the subject patent, add the priority date, and use semantic search to surface patents that are potentially relevant to what you are investigating in this AI patent search tool. You may still have hundreds of results to review, but they are ranked around the technical concepts in your query, giving you a much more focused place to start.

And as this test showed, you do not necessarily have to rely only on the exact references in the initial results. Features like “More like this” can help you follow a promising patent into closely related applications and references that may be worth investigating further.

The beauty of PQAI is that you can export the results and review exactly what the search surfaced. That also makes tests like this easier to reproduce and evaluate independently.

Want to run the search yourself? Try PQAI for free and see what it surfaces for your own patent search. No credit card required.