|By Russ Still
April 09, 2020
The age-old practice of memorizing FAA knowledge test questions is becoming a really bad idea. This change represents a complete philosophical reset for many test-takers and highlights the FAA's new doctrine: learn the concepts.
In January, 2020, the FAA announced its new contractual partnership with testing center company, PSI. What it said suggests that major changes are coming to all airman knowledge tests, possibly at the most fundamental levels. Not only are test questions and answers changing, the basic fabric of the multiple choice test may be restructured altogether. We can argue whether the changes are good or bad, but the result is still the same - if the FAA and PSI stick to the plan, rote memorization will become a useless test-prep strategy.
In the old world, the FAA's Airman Testing Branch (AFS-630) dictated what questions appeared on the various knowledge tests. It was a fairly straight-forward process. They wrote the questions, picked three answer choices, and updated the associated charts and figures in the Testing Supplements. They even published the full databases of questions and answers. If you go back far enough - 1977 in my case - they even published their own Written Test Guide which gave you all the questions and answers in a nice, printed book. With time, however, came changes. In 2008, the FAA stopped releasing the question databases. The PTS became the ACS, and the Computer Testing Supplements became the Knowledge Testing Supplements. And possibly the biggest changes yet to the testing process is now on the horizon.
In the January announcement, the FAA formally declared its new partnership with PSI Services, LLC. The contract's term was ten years and the relationship was given a name: ACTS - Airman Certificate Testing Service. It defines how Knowledge Tests will be designed and delivered in the future. And there is probably some expediency needed. The old testing software (IDS) is no longer supported by the software developer. That creates a clear need for a new system, but the changes that are anticipated will have a profound impact on the ways that aviators study for Knowledge Tests.
The FAA, along with many educators, has long expressed concerns about rote memorization. It has become the most common strategy for test preparation and defeats the goal of testing - to ensure that applicants have attained a certain level of knowledge.
Many third-party companies have developed products that leverage rote memorization instead of conceptual learning. For them, that will be a problem in the new testing world. Memorization and memory-tricks just won't do it anymore. Test prep programs with household names today may become quickly obsolete.
The FAA has explicitly stated that they want to reduce the effectiveness of memorization and additionally, thwart "question harvesting". The days of knowing the questions in advance may be coming to an end. Digging through books of sample questions and taking endless rounds of practice quizzes will likely become ineffective ways to prepare for a test. Your best defense today is to use training products and programs that transfer knowledge to you, not answers to test questions.
Learning the concepts instead of the actual questions/answers will become the best way to score well on future Knowledge Tests. And that is a good thing. Understand the concepts and it doesn't matter how the individual questions are worded.
So, in future exams, expect to see lots of questions you've never encountered before in any practice test. New questions should be no more difficult to solve, but they probably won't be exact matches to questions you practiced with. Ultimately, the FAA may even discontinue use of the Knowledge Testing Supplements. When you encounter a METAR or TAF on a future test, it may be one you've never seen before, popped up onto the computer monitor. But again, learn to interpret these reports and it doesn't matter which ones appear on your test. The demise of the Knowledge Testing Supplement is a logical move if the FAA wants to reduce the reliance on memorization during tests. That is highly likely to be in our future.
But wait, there's more.
Not only will the questions/answers be in a constant state of change, but the actual structure and function of the test itself may become fundamentally different. The new system is expected to support these type of advanced features:
1. Automatic Item Generation
3. Video and Audio Capabilities
4. Scenario-based Testing
At this point, little is known how these advanced ideas may appear on future tests. But it strongly hints that Knowledge Tests may become more than simple multiple choice exercises. Will everything be on-screen during the test? Will some questions be asked verbally in video clips? Will the Knowledge Testing Supplement go away?
The hope is that these new features may actually make tests more relevant, and taking them maybe even entertaining. It will be very interesting to see how this plays out. But without a doubt, testing will be different.