How do I disable the 'uncategorized' block pattern category in WordPress?

I want to disable the uncategorized block pattern category in WordPress, how do I achieve this?

I have used the following code and it doesn't work, what do I do, any suggestions would be appreciated:

function remove_uncategorized_pattern_category() {
    unregister_block_pattern_category( 'uncategorized' );
}

add_action( 'init', 'remove_uncategorized_pattern_category' );