What does "Malicious Models" mean?
Table of Contents
Malicious models are modified versions of computer programs that can behave in harmful ways while appearing to be safe. These models often start as normal, well-functioning programs, but someone can change their design to include harmful features.
How They Work
Changing the Model: A harmful model can be created by training it on tasks that aim to trick or deceive users. This modified model can look completely normal to most users.
Quantization: One common technique is to reduce the size of the model by using fewer bits to store its data. This process, called quantization, makes it easier to run on everyday devices. However, it can also hide the harmful changes made to the model.
Attacks During Use: Once a harmful model is released, attackers might use methods to change just a few bits of data in the model while it is being used. Even a single small change can turn a regular model into one that acts unpredictably or maliciously.
Real-World Risks
Malicious models can be shared on popular platforms where many people download tools for various tasks. This can expose users to unexpected and harmful behavior without them realizing it. It is essential to be cautious about the source of any model and how it was created to avoid potential dangers.