FAA Knowledge Tests Are Changing - Here's What You Need to Know.
by Russ Still
Published
Updated
Some of the biggest changes in decades are coming to the airmen knowledge tests. If there’s one in your future, it’s time for you to get on board. There are years of inertia to overcome. Let’s start with some backstory.
The FAA has offered multiple choice Knowledge Tests since the 1970s. Since then, they have also been publishing their bank of test questions and answer choices. You still had to figure out which was the right answer, but wow, what a concept. Get to see your exam in advance! That should make it easy, right? Well, yes. In fact, too easy.
2008 – FAA Stops Publishing Test Questions
At some point, the FAA determined that tests didn't accurately evaluate student learning. Memorizing the tests in advance was certainly a big part of the problem. So, in 2008, the decision to stop publishing the testing databases was made. Test prep providers had the last published database, but they’d have to use other means to keep their questions current over time. There were a number of ways to do that.
How Do Providers Keep Up?
First, the test prep providers simply watched the FAA documents for changes. These included the various PTS and ACS documents. If a change appeared in one of those, there was a great chance that something new would appear in the FAA exam. The FAA handbooks would also be updated every few years. They were another good harbinger of change.
The FAA questions frequently referred to graphics. These figures were published in books called the Knowledge Testing Supplements. They were updated every few years. As long as you used the latest version of the Supplement, you were guaranteed to see the sectional chart, weather report, or other graphic that would appear in the test. For example, if you checked the Supplement, you knew exactly which METARs would appear on the exam. If a new version of the Supplement came out with a different figure for the METARs, it was a sure bet that any METAR questions had to change.
The FAA published, at least from 2017, a series of reports called What's New and Upcoming in Airman Testing. These were excellent resources for test prep providers. When the FAA announced, for example, that there would be no more ADF/NDB questions, most learned about it in the document. The What’s New document quietly disappeared at the end of 2021, but a year later (January 2023) reappeared as the Airman Testing Community Advisory. Every test prep provider now keeps a close eye on this one for news. But where test questions are concerned, the Advisory still only provides generalities. It takes a lot of experience and skill to generate new sample test questions from the slim information it provides.
With the ubiquity of social media, it’s easy to find people discussing their knowledge test experiences, after the fact. This is not a practice that the FAA is fond of, but it’s still a source of information for some test prep providers. If a particular provider is known for having nearly 100% of actual test questions, it may be an instance of test takers memorizing questions and reporting them back to the provider, possibly for some type of incentive. The FAA calls this “question harvesting”, and it’s abundantly clear why this would annoy them. In case you haven’t gotten the memo, the FAA wants neither you, nor the test prep companies, to know what the questions are. And that's really what this whole thing is about.
The Problem With Memorization
The Aviation Instructor’s Handbook lists the four Basic Levels of Learning. First, and most fundamental, is “rote”, as in rote memorization. At this level, the student has the ability to recite memorized data, without any deep understanding.
Examples of valid rote items include things like regulations, fuel and electrical systems, airspace designations, and V-speeds for a particular airplane. When we talk about rote memorization, we are NOT discussing the memorization of test questions and answers. This is a fundamental distinction that is frequently misunderstood, even by flight instructors. Rote learning in this context does NOT refer to memorizing test questions. If it did, the FAA wouldn’t go to such extremes to prevent people from doing it.
2020 – FAA Announces Changes
In 2020, the FAA and PSI (the company that produces and delivers the tests) announced a new 10-year contractual arrangement covering Airman Knowledge Tests. The framework was called the ACTS, standing for Airman Certificate Testing Service. I announced this in an April 2020 Gold Seal article. Give it a look. The details are worth knowing.
One of the ACTS primary goals was to prevent “question harvesting”. Remember that from a few paragraphs back? If the FAA can prevent test prep providers from knowing what the real FAA questions are, then the act of memorizing questions becomes moot. And that is exactly where we are headed. No reason to worry whether your practice tests use “real” questions. The providers would have no way of knowing what the real questions are.
Now, before your heart starts racing, understand that the topics on the test will still come straight out of the ACS. The FAA isn’t trying to trick you. They just don’t want you to memorize the tests. They want to ensure that you can answer questions because you have reached an acceptable level of learning. And those are the levels in the pyramid – understanding, application, and correlation. If you simply memorized the test questions, you’d never reach those levels.
This brings us to the actual changes in the entire testing philosophy: stop question harvesting so that students cannot have access to practice tests that use the real FAA questions.
| You might be surprised to hear this, but the FAA doesn't have a single database of questions. Well, it does and it doesn't. There is a database, one for each certificate or rating, but they are further broken up into templated test files. Questions can be removed from the master list, but still appear in existing compartmentalized files. Feel free to figure that one out. I'm a software guy and I still don't fully understand how it works or why it was designed like that. But be advised that changing and adding questions is not just a simple update function. |
2023 – Airman Knowledge Tests
The first step taken to slow the harvesting of test questions was actually very simple and low tech. Just make changes to the questions in the database more often. As I just mentioned, though, it's easier said than done. Nonetheless, in 2023, The FAA and PSI began updating groups of questions roughly every quarter. Now, I’m not saying that the entire question list was swapped out; just that a lot of new questions appeared and a lot of old questions disappeared. It was the first step in the process.
Additionally, in April, the tests began including five extra questions that were ungraded. Whether you got them right or not had no effect on your score. You didn't even know which ones they were. The FAA referred to these as “validation questions”. They were potential test questions that were being evaluated. If a question was missed at a statistically unacceptable rate, back to the drawing board it went. Otherwise, it might be added to the PSI bank of available questions.
2025 – Question Figures Go Online
Preparing to do away with the Testing Supplements, PSI began taking the existing question figures and cropping them down for display online, within the questions themselves. The figures effectively became part of the questions so that everything was self-contained onscreen. It’s exactly what Gold Seal has been doing all along.
2026 – The Time Has Arrived for Big Changes
In the spring of 2026, the FAA announced that the new testing policies were ready for rollout. The various Knowledge Testing Supplements were being retired. The first two to go were for the Instrument Rating (IRA) and the Remote Pilot Certificate (UAS). On October 27, the two books, previously supplied at the testing center, went “poof”.
Since the question figures were already loaded into the questions, there was no longer a need to publish the Supplements. This means every figure could change as frequently as the FAA wished. No test providers would know what any of the figures were; the FAA and PSI would change them too quickly to identify. I can’t think of an easier way to juggle the questions around and keep them secret.
What Comes Next in FAA Testing?
The frequent changes to the test questions, along with the retirement of the Testing Supplements, are really just first steps. Sooner or later, every existing test question will have been swapped out. And some of the questions will be computer-generated based on templates. This method can produce countless new questions on a single topic, each containing different parameters and variables.
These dynamically produced questions can increase the size of the question database massively and with less human involvement (and less cost). PSI calls this “AIG” for Automatic Item Generation. No one, not even the most famous test prep “brand”, will be able to keep up with practice tests any longer. In 2025, PSI indicated that this was "just around the corner." A year later, we're still waiting, but it could be very soon.
We should anticipate a further expansion of this AIG function in the future. This will include questions created at runtime. They will be AI-generated and only exist for a single test. In theory, each of these questions might only be seen by one person and afterwards disappear forever. For these dynamic AIG questions, it will be impossible to know what they are in advance. They won't exist in anyone's practice tests. But you know what? Learn the material, not the questions, and everything will be perfectly all right.
No matter whose product you use for test prep, I cannot stress this enough. DO NOT MEMORIZE questions and answers. It’s highly UNLIKELY that you’ll see them exactly the same on the FAA exam. Some questions will be completely new, while others may be reworded and include different variables. In any case, you won’t be able to memorize your way to victory.
Some Tactics To Get You a Higher Score
In your ground school, focus on the lessons more than the quizzes. That’s where the learning occurs. The quizzes only exist for you to see how effectively you studied. Think of them as thermometers, checking the temperature along the way.
Avoid taking quizzes over and over. If your scores do improve, it’s not because you’ve had some spark of enlightenment. You simply memorized the questions by virtue of seeing them over and over. In fact, doing this, you’ll get to the point where you can pick the right answer without even reading the question. A cool idea, perhaps, but it creates a false sense of security. It is useless in your quest to become an educated pilot.
Gold Seal has quizzes at the lesson and stage levels, and also full course practice exams. Each one is archived for later use. (Check your "Progress Report" page.) Each lesson quiz is delivered in a random order, and every course practice exam contains questions randomly selected from the full database. Evey quiz and exam is different. But again, use them for evaluations, not as learning tools.
Aviation isn’t golf. I know 70% is passing, but don’t shoot for a low exam score. 70% is passing, but don’t kid yourself. It’s also almost failing.
Some people will tell you that you shouldn’t score too high. A checkride examiner might drill you extra hard to make sure you didn’t cheat! This is pure nonsense. Don’t buy it. An examiner can tell within the first five minutes of your oral exam whether or not you know your stuff. Study hard and strive to learn as much as you can. And that’s what will earn you confidence and a great score.
