Strictly following pattern or modifying it to fit your needs

I am working on a mid-sized project and I am searching for a fitting structural pattern. But nothing really fits my needs 100%. Is there a rule that states that you should strictly follow patterns, even if it increases complexity in your projects? Or should I take some liberties in following the pattern to make the structure of the project less complex?