In the first seminar assignment, your goal is to use genetic algorithms to find a path out of a maze,
represented as a vector of strings, where $\#$ characters represent walls, $.$ represent empty spaces, and
S and E represent the starting and ending points, as in a given example below:
\begin{center}
\begin{BVerbatim}
maze = c("####E######",
"##...#.####",
"#..#.#.####",
"#.##...####",
"#.##.#..S##",
"###########")
\end{BVerbatim}
\end{center}
You can move through the maze in four directions, left, right, up, and down. In the example above,
the shortest path from the starting position S to the exit E is composed of the following moves: left,
left, up, left, left, up, up, up. In your solution, this should be represented as a string ”LLULLUUU”.
Your task is to create a function that will be able to find path as short as possible out of any maze
represented in such a way
\section{Solution}
\subsection{Task 1}
I decided to write this assignment in python using the pygad library becouse I am more familiar with this programming language.
\subsubsection{Task description}
Create a function that reads the 2D representation of a maze and returns the shortest path found by
a genetic algorithm. To do this, you will need to:
\begin{itemize}
\item Read the map into a suitable format (for example, a matrix).
\item choose a suitable representation of your solutions (the path). Hint: you don’t need to use strings
when working with the genetic algorithm. You can use numeric or binary representations for the
GA function and then convert the result to a string as the final result.
\item Define the fitness function. Make sure to penalise paths through walls - those are invalid solutions
\item Run the genetic algorithm with suitable settings.
\end{itemize}
\subsubsection{Read the map into a suitable format}
I decided to read all the maps provided in the assignment into a list of lists. Each list represents a
maze and each element of the list represents a row of the maze.
\subsubsection{Choose a suitable representation of your solutions}
I decided to use a binary representation of the solution same size as an original maze. Each bit represents a move. 0 means that the agent did
not visit the cell and 1 means that the agent visited the cell. So if maze is of size N x M, the solution will be of size N x M. But there is no
such thing as N-dimensional array that GA accepts. So I reshaped the matrix into a vector of size N * M and worked with that kind of solution.
\subsubsection{Define the fitness function}
This part was the most difficult for me. I maybe overcomplicated that part but at least it yields good results.
Before runing the algorithm I have decided to construct a punish matrix, which is a matrix of the same size as the maze. Each cell in the punish matrix is
evaluated before the algorithm starts. The evaluation is based on the position of walls and valid paths. So if there is a wall in the cell, the fitness value in that cell is set to some low scalar.
If there is a valid move then the fitness value in that cell is high. So everytime the fitness function is called, the matrix product will be executed and some initiall fitness value will be computed asfollows:
But though experimentation I found that this approach was not good enough so I modified the function by adding punsihment if the agent did not start at the starting position and
if the agent did not end at the ending position. Still the results were not good enough so I decided to check if there is a valid path from the starting position to the ending position.
If there is no valid path then I would punish the agent otherwise I would give him some reward. This approach yielded better results.
But still I was not satisfied with the results so I decided to add some more punishes and rewards:
\begin{itemize}
\item Add a reward if agent finds a shorter path than the best path found so far.
\item Update weights in punish matrix so that the agent will prefer to move on best path found so far.
\item If the agent does not find any valid path until 80\% of the GA iterations then activate critical search phase. That means that the agent
will be rewarded if he finds \textbf{any} path from start to end, even if it maybe isn't the correct one. This way the weights are updated
Other mazes found also found some solutions, but they were not optimal. Or they were trying to go through a wall becouse the critical section was activated. I think that the problem is that the mutation and crossover operators are not good enough.
I initially used the random mutation type provided by library pygad. It was not good enough for this task, because it was not taking into account the walls.
So I redefined mutation function in such way, that we add random bits where there is no wall. If there is a wall, we set random number of bits to 0.
The function is defined as follows:
\begin{small}
\begin{center}
\begin{verbatim}
def on_mutation(generations, ga_instance):
maze = mazes[maze_ix]
# Firtly find the instances where there are no walls
I also generated the initial population using the same function, but firstly I generated some random bitarrays and then applied the same function to them.
The results I got using this approach were suprising! I got a perfect score on all mazes. I think that the reason for this is that the mutation function is not only taking into account the walls, but also the previous solution. The algorithm converges really fast now. In 5 generations of 400 specimens we get a shortest path! I even generated a 1000 x 1000 maze and it solved it in around minute!
For treasures to be found I had to modify my fitness function, initial population generation and mutation function. I introduced clustering of cells which are close to the treasures.
That way the genetic algoritm will also mutate in that way. Clusterization is done using simple alogirthm, which just checks for K valid cells arround the treasure and adds them to
the cluster. The cluster is then used to influence generation of the initial population and mutation of the population.
if maze.shortest_path == [] and path_len > 0 and treasures_found >= len(maze.treasures) // 2:
fitness += treasures_found * 1000
print('First path found')
maze.shortest_path = path
maze.treasures_found = treasures_found
maze.adjust_weights(complete_path)
#Check if the current path is shorter than the shortest one
elif treasures_found > maze.treasures_found and path_len > 0:
fitness += 1000 * treasures_found
print('Path with more treasures found!')
maze.shortest_path = path
maze.treasures_found = treasures_found
maze.adjust_weights(complete_path)
elif path_len < len(maze.shortest_path) and treasures_found > maze.treasures_found and path_len > 0:
fitness += 1000 * treasures_found
print('Path with less steps found!')
maze.shortest_path = path
maze.treasures_found = treasures_found
maze.adjust_weights(complete_path)
..........................
\end{verbatim}
\end{center}
\end{small}
So now each time there is a treasue in the path or if there is a shorter path, the fitness function will increase the fitness of the specimen. The fitness function also takes into account the number of treasures found. If there is a path with more treasures found, the fitness function will increase the fitness of the specimen.
And the mutation function is same as in task 2, but now it also takes into account the treasures. The mutation function is defined as follows: