Knowledge Test Prep - Why Memorization is a Bad Idea | GroundSchool.com

Knowledge Test Prep - Why Memorization is a Bad Idea

The age-old practice of memorizing FAA knowledge test questions is becoming a really bad idea. This change represents a complete philosophical reset for many test-takers and highlights the FAA's new doctrine: learn the concepts.

In January, 2020, the FAA announced its new contractual partnership with testing center company, PSI. What it said suggests that major changes are coming to all airman knowledge tests, possibly at the most fundamental levels. Not only are test questions and answers changing, the basic fabric of the multiple choice test may be restructured altogether. We can argue whether the changes are good or bad, but the result is still the same - if the FAA and PSI stick to the plan, rote memorization will become a useless test-prep strategy.

How Did We Get Here?

In the old world, the FAA's Airman Testing Branch (AFS-630) dictated what questions appeared on the various knowledge tests. It was a fairly straight-forward process. They wrote the questions, picked three answer choices, and updated the associated charts and figures in the Testing Supplements. They even published the full databases of questions and answers. If you go back far enough - 1977 in my case  - they even published their own Written Test Guide which gave you all the questions and answers in a nice, printed book. With time, however, came changes. In 2008, the FAA stopped releasing the question databases. The PTS became the ACS, and the Computer Testing Supplements became the Knowledge Testing Supplements. And possibly the biggest changes yet to the testing process is now on the horizon.

FAA and PSI Collaboration

In the January announcement, the FAA formally declared its new partnership with PSI Services, LLC. The contract's term was ten years and the relationship was given a name: ACTS - Airman Certificate Testing Service. It defines how Knowledge Tests will be designed and delivered in the future. And there is probably some expediency needed. The old testing software (IDS) is no longer supported by the software developer. That creates a clear need for a new system, but the changes that are anticipated will have a profound impact on the ways that aviators study for Knowledge Tests.

FAA ACTS

Expect These Changes

The FAA, along with many educators, has long expressed concerns about rote memorization. It has become the most common strategy for test preparation and defeats the goal of testing - to ensure that applicants have attained a certain level of knowledge.

Many third-party companies have developed products that leverage rote memorization instead of conceptual learning. For them, that will be a problem in the new testing world. Memorization and memory-tricks just won't do it anymore. Test prep programs with household names today may become quickly obsolete. 

The FAA has explicitly stated that they want to reduce the effectiveness of memorization and additionally, thwart "question harvesting". The days of knowing the questions in advance may be coming to an end. Digging through books of sample questions and taking endless rounds of practice quizzes will likely become ineffective ways to prepare for a test. Your best defense today is to use training products and programs that transfer knowledge to you, not answers to test questions.

FAA Rote Memorization


Learning the concepts instead of the actual questions/answers will become the best way to score well on future Knowledge Tests. And that is a good thing. Understand the concepts and it doesn't matter how the individual questions are worded. 

So, in future exams, expect to see lots of questions you've never encountered before in any practice test. New questions should be no more difficult to solve, but they probably won't be exact matches to questions you practiced with. Ultimately, the FAA may even discontinue use of the Knowledge Testing Supplements. When you encounter a METAR or TAF on a future test, it may be one you've never seen before, popped up onto the computer monitor. But again, learn to interpret these reports and it doesn't matter which ones appear on your test. The demise of the Knowledge Testing Supplement is a logical move if the FAA wants to reduce the reliance on memorization during tests. That is highly likely to be in our future.

But wait, there's more.

Not only will the questions/answers be in a constant state of change, but the actual structure and function of the test itself may become fundamentally different. The new system is expected to support these type of advanced features:

1. Automatic Item Generation
2. Drag-and-Drop
3. Video and Audio Capabilities
4. Scenario-based Testing


At this point, little is known how these advanced ideas may appear on future tests. But it strongly hints that Knowledge Tests may become more than simple multiple choice exercises. Will everything be on-screen during the test? Will some questions be asked verbally in video clips? Will the Knowledge Testing Supplement go away?

The hope is that these new features may actually make tests more relevant, and taking them maybe even entertaining. It will be very interesting to see how this plays out. But without a doubt, testing will be different.

 

 



Russ Still

About the Author

Russ Still is a career flight instructor and ATP. He has written numerous articles for technology and aviation publications and authored “The Unbroken Chain” in 2001 (ISBN-10: 189652284X). Russ is an 8-time Master CFI and founded the Gold Seal Aviation Training Network in 2003. He holds a Bachelor of Science in Computer Science from the University of Florida.