Parameter object slows down program significantly in C

I have a bunch of related arrays arr1, arr2, arr3. I pass these to a few functions, like foo.

int foo(int* arr1, int* arr2, int* arr3, ...) { ... }

The argument list starts to get pretty long, so I wanted to make a struct Bar to collect these related arrays in a single struct, like so:

struct Bar {
    int* arr1;
    int* arr2;
    int* arr3;
};

This allows me to simplify foo into foo(struct Bar bar, ...) { ... }, which is great. But when I do this the execution time goes from 1m35 to 2m18, which is a slowdown 45%. Using pointers instead, like foo(struct Bar* bar, ...) is faster at 2m03, but still slower overall. All of these measurements were taken with gcc 12.2.0. I compiled an optimized build (-O3).

I understand that adding a layer of indirection is bound to slow the program down, but given that this is such a common pattern and the change is so slight, I expected that the compiler would optimize this indirection away.

I also wonder if there is anything I can do to tell the compiler what I'm doing. Kind of like how inline can be used to change how functions are compiled. If nothing else, I'm curious as to why this is apparently a hard thing for the compiler to recognize and optimize.

Thank you in advance!

P.S. Here's the full code, it's short enough to put here. It's before I added the struct and it finds a solution to the N queens problem on the torus. The three arrays I'm trying to put into a struct are cols, updiags, downdiags.

#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#define N 31

int upDiag(int row, int col) {
    int updiag = row + col;
    if (updiag >= N)
        updiag -= N;
    return updiag;
}

int downDiag(int row, int col) {
    int downdiag = row - col;
    if (downdiag < 0)
        downdiag += N;
    return downdiag;
}

bool isSafeTorus(int* cols, int* updiags, int* downdiags, int row, int col, int updiag, int downdiag){

    for(int i = 0; i < row; i++) {
        if (cols[i] == col || updiags[i] == updiag || downdiags[i] == downdiag) {
            return false;
        }
    }
    return true;
}

bool solveNQUtil(int* cols, int* updiags, int* downdiags, int row){
    /* If all queens are placed then return true */
    if (row >= N)
        return true;
    /* try placing this queen in all coloms one by one */
    for (int i = 0; i < N; i++) {
        /* Check if the queen can be placed on board[row][i] */
        int updiag = upDiag(row, i);
        int downdiag = downDiag(row, i);

        if (isSafeTorus(cols, updiags, downdiags, row, i, updiag, downdiag)) {
            cols[row] = i;
            updiags[row] = updiag;
            downdiags[row] = downdiag;
            /* place rest of queens */
            if (solveNQUtil(cols, updiags, downdiags, row + 1))
                return true;
            /* If placing queen in board[i][col] no solution, remove queen*/
        }
    }
    /* Queen can not be placed this row */
    return false;
}


void main(){
    int* cols = (int*)malloc(N * sizeof(int));
    int* updiags = (int*)malloc(N * sizeof(int));
    int* downdiags = (int*)malloc(N * sizeof(int));

    if (solveNQUtil(cols, updiags, downdiags, 0) == false) {
        printf("Solution does not exist");
    }

    for(int i = 0; i < N; i++) {
        printf("%d\n", cols[i]);
    }
}