References

Bevilacqua, F. (2018, October 29th). https://gamedevelopment.tutsplus.com/. Retrieved from Finite-State Machines: Theory and Implementation: https://gamedevelopment.tutsplus.com/tutorials/finite-state-machines-theory-and-implementation--gamedev-11867

D, S. (2018, Novermber 5th). Teaching An AI To Play A Simple Game Using Q-Learning. Retrieved from Practical Artificial Intelligence: https://www.practicalai.io/teaching-ai-play-simple-game-using-q-learning/#

Gallear, W. (2018, October 08). A.I Techniques. Retrieved from A.I And Games: https://prezi.com/view/GsEiZBJDMchJZzfCNaHx/

Gallear, W. (2018, October 29th). Decision Making Concepts. Retrieved from A.I And Games: https://prezi.com/view/GsEiZBJDMchJZzfCNaHx/

Lent, T. (2018, October 1st). New Ways Artificial Intelligence Can Improve Player Experience in Games. Retrieved from https://www.linkedin.com/pulse/new-ways-artificial-intelligence-can-improve-player-experience-lent: https://www.linkedin.com/pulse/new-ways-artificial-intelligence-can-improve-player-experience-lent

Moss, R. (2018, October 8th). 7 examples of game AI that every developer should study. Retrieved from https://www.gamasutra.com/view/news/269634/7_examples_of_game_AI_that_every_developer_should_study.php: https://www.gamasutra.com/

Reynolds, C. (2018, October 08). Boids. Retrieved from Background and Update: https://www.red3d.com/cwr/boids/

Russell, B. (2018, October 1st). Fortnite System Requirements: Can My PC Run Fortnite? Retrieved from Game Revolution: https://www.gamerevolution.com/guides/398037-fortnite-system-requirements-can-my-pc-run-fortnite

Techopedia. (2018, November 5th). Artificial Neural Network (ANN). Retrieved from https://www.techopedia.com/definition/5967/artificial-neural-network-ann: https://www.techopedia.com/definition/5967/artificial-neural-network-ann

Techopedia. (2018, October 29th). https://www.techopedia.com/definition/1809/fuzzy-logic. Retrieved from Fuzzy Logic: https://www.techopedia.com/definition/1809/fuzzy-logic

Watkins, C. (1989). Learning from Delayed Rewards. Cambridge: University of Cambridge.

Learning Techniques

Learning Techniques:

Q-Learning:

Q-Learning is a reinforcement learning technique that is used in machine learning. It is an algorithm that that does not need a model of its environment and can be used on-line. (Watkins, 1989) Therefore, Q-Learning is very suited for repeated games against an unknown opponent. This is mainly used in pathfinding where the reward is higher for points connected to the goal. This basically means an AI with Q-learning will keep going through this process via trial-and-error until it finds the optimal route/the highest reward. In laymen terms, the basic premise is that the algorithm is taught to take certain actions based on prior experience by rewarding or punishing actions. Similar to teaching a dog to sit by giving it treats for good behaviour. (D, 2018)

Artificial Neural Networks:

An Artificial Neural Network (ANN) is a computational model based on the structure and functions of biological neural networks. Information that flows through the network affects the structure of the Artificial Neural Network because a neural network changes - or learns, in a sense based on that input and output. An Artificial Neural Network has several advantages but one of the most recognized of these is the fact that it can learn from observing data sets. In this way, Artificial Neural Networks are used as a random function approximation tool. (Techopedia, 2018)

In terms of games, an Artificial Neural Network works by taking information and then processes that information through hidden layers, which it then translates and decides what the AI needs to do next in response. Basically, it gives the AI its own thinking brain and it is used to think just like a human player. Additionally, it has its own decision tree and it can learn from its previous experiences, for example; For example, if the Enemy AI is wandering around and bumps into the Player AI it’ll then process different actions it can take and decides upon an outcome on how to proceed.

Tactics & Strategy:

Tactics & Strategy:

The difference between strategy and tactics is often described as the following; Strategy is Long-Term, Tactic are Short-Term. In terms of Artificial Intelligence, AI Tactics and AI Strategies are a set of Algorithms and Decisions set to the games AI that allows them to make basic decisions. AI Techniques such as AI Tactics and Strategy is important as it drastically improves the player experience a lot and will make the player think themselves as to how to deal with the situation.

AI Strategy:

AI Strategy is an AI Technique used in a lot of Strategy-based games, such as Real-Time Tactics, First-Person/Third-Person and Roleplay, as it uses an Algorithm rule put in place on a set of characters in that scene. For example, in First-Person Shooters like Call of Duty or Battlefield there could be a group of AI enemies tasked to kill the Player Character and will achieve this as an Enemy Unit. Each AI Enemy would have its own role to play and have their own decisions to make. They’ll make use of their own algorithm that affects the way they move around the environment. Overall, Enemy AI will have their decisions made by a group strategy, which can mean that they can team up together to overwhelm the player and eliminate them.

AI Tactics:

AI Tactics is an AI Technique that is linked to AI Strategy; For example, Enemy AI in a group will work together and follow a unique set of commands thanks to its algorithm’s tactics, with each Enemy AI given a job that’ll suit the AI’s role. This means, the “Leader” of the AI will sit back and give commands to the others, the “Heavy” will take the lead and gun down the Player, the “Medic” will go around and heal its allies, and finally the other “Soldiers” will act as support, protect the leader and shoot the Player. By using Algorithms, each AI will work on its own Decision Tree and own State Machine, meaning that they’ll think for themselves based on the decisions they make, which’ll make an impact on the other AIs in the group. This overall behaviour and AI duties will rely on what the unit is doing as a whole, if the unit is programmed to attack the player character then all AI members will do so.

Decision Making Concepts:

Decision Trees:

Decision trees are evaluated from root to leaf, every time. For a decision tree to work properly, the child nodes of each parent must represent all possible decisions for that node. If a node can be answered "Yes”, “No” and “Maybe", there must be three children, Yes node, No node and Maybe node. This means there's always some lower node to traverse, until reaching an end node. (Gallear, Decision Making Concepts, 2018)

Decision Trees are extremely helpful in the way that they work and are often found in games where decisions are rather basic, as adding too many decisions may result in an AI having problems if in large groups. Additionally, Decision Trees help by making AI feel more real by allowing them to make choices similar to what a player would make. For example; Can it see the player? If yes, then it can give chase or attack. If no, then it can continue on its patrol route. And if maybe it may go into an alert state and checkout what the disturbance was.

State Machines:

A finite-state machine is a model used to represent and control execution flow. It is perfect for implementing AI in games, producing great results without a complex code. Only a single state can be active at the same time, so the machine must transition from one state to another in order to perform different actions. FSMs are commonly used to organize and represent an execution flow, which is useful to implement AI in games. The "brain" of an enemy, for instance, can be implemented using an FSM: every state represents an action, such as attack or evade, or even block if capable. (Bevilacqua, 2018)

From the above Bevilacqua's FSM you can see the fundamentals of how the algorithm works, this is the state machine for an AI character. Now the state machine can work in relation to a health bar, for example if the health bar gets below 60% for the AI then the AI would decide to hide or take cover to reduce the risk of taking more damage and reducing health. When the health bar drops again below a certain point, the AI would choose to run from the scene and works similarly with the Fuzzy Logic algorithm whose state membership has a float range and not made up of 0 and 1's.

Fuzzy Logic:

Fuzzy Logic is a logic operations method based on many-valued logic rather than binary logic (two-valued logic). Two-valued logic often considers 0 to be false and 1 to be true. However, Fuzzy Logic deals with truth values between 0 and 1, and these values are considered as intensity (degrees) of truth. Fuzzy Logic may be applied to many fields, including control systems, neural networks and artificial intelligence (AI). (Techopedia, 2018)

In terms of gameplay, Fuzzy Logic is used as a platform to make decisions to determine an AIs state to give states to an AI. Usually in older games there would be only two states: Alive and Dead. For example, in Donkey Kong (1981) Jumpman (aka, Mario) would alive until they bumped into an AI, and at that point it would be all over, and they were dead. As mentioned above, this application of fuzzy logic is expanding this linear system by introducing new waypoints between 0 and 1.

In games like Elder Scrolls: Skyrim, Enemy AI will start out as aggressive when their health bars are full or near full. Upon reaching a certain percentage of health they will start to flee and stay away from the Player until they’ve recovered enough health. These are states determined by fuzzy logic which, when the AI drop below a certain percentage of their maximum health, tells them that they are in some sort of unhealthy state, and must retreat.

AI Techniques

AI Techniques For Games:

Boids Flocking:

In the natural world, organisms exhibit certain behaviours when travelling in groups. This phenomenon, also known as flocking, occurs at both microscopic scales (bacteria) and macroscopic scales (fish). (Gallear, A.I Techniques, 2018) It is based on three-dimensional computational geometry of the sort normally used in computer animation or computer aided design. The basic simulated flocking creatures were designated “Boids” and the standard flocking model is made up of three steering behaviours which detail how an individual “Boid” manoeuvres based on the positions and velocities of its nearby flock mates. (Reynolds, 2018). By using computers, these patterns can be simulated by creating simple rules and combining them. This is known as emergent behaviour and can be used in games to simulate chaotic or lifelike group movement. (Gallear, A.I Techniques, 2018). This technique works on three simple components known as cohesion, separation and alignment.

Cohesion:

A flock is when a group of boids all stay close together. Cohesion is when each boid will look at the position of every other boid to see if it is within the neighbour radius. If a fellow boid is within the radius than it becomes a flock mate. It than takes the average position of each boid and will steer toward that position. This makes it so each boid is trying to get to the centre resulting in them staying together. I.E: Steer to move toward the average position of local flockmates.

Separation:

Even though each boid is endeavouring to draw near together you don't need them covering or hitting against as this doesn't this doesn’t simulate real life which is the place the partition some portion of the calculation comes in. They do this by having each boid keep a specific measure of room in the middle of one another. In the event that a boid gets excessively near another boid than it will add a converse speed to itself to go the other way. I.E: Steer to avoid crowding local flockmates.

Alignment:

This is the part of the algorithm which oversees the direction of the flock. The way it works is that every frame each boid will look at the heading it which it is heading and those of its flock mate’s aka those inside neighbour radius. It will than realign itself to match the heading of all its neighbours. It than averages the velocity vectors of those in the neighbour radius and the resulting vector point is the way than the boid will head towards. I.E: Steer towards the average heading of local flockmates.

Pathfinding Concepts:

A* Algorithm:

In Computer Science, A* is a computer algorithm that is widely used in pathfinding and graph traversal, which is the process of finding a path between multiple points, called "nodes". It enjoys widespread use due to its performance and accuracy. In Game Design, most users will use the A* Algorithm to perform their pathfinding methods. While this is seen as the most popular, but this doesn’t that it is the best is best solution for every situation. A* Algorithm’s pathfinding combines the concept of Heuristics with a simple Algorithm to generate a path for the AI to work. A* works on these Algorithms to generate numbers and will take the path with the lowest number being the better route.

In Pathfinding the location, the AI can move to and from is known as nodes. Each node has a cost to it (look at table). To walk from one node to another it has a cost known as the route cost. When starting the pathfinding will use the first algorithm to generate a heuristic cost which is then used in the second algorithm. To further example here’s a quote to explain how the A* Algorithm works. A* has many advantages to it. The main advantage is that due to heuristic function it can quickly recalculate if the goal is moving and will only check nodes that need to be checked making it super responsive and is the reason why it most popular. A* is a super compact pathfinding that will find the best path based on heuristics however this pathfinding has a weakness of that it will always try to find the best route when sometimes you may want the AI to flank round or maybe being sneaky behind a wall instead of trying to get around the wall.

Dijksta’s Algorithm:

Dijkstra's Algorithm (Also called Uniform Cost Search) prioritizes which paths to explore. As opposed to exploring all possible paths, it prefers lower cost paths. It allows for assigning lower costs to encourage moving on roads and higher costs to avoid environments such as forests which could have more obstacles, using higher costs to discourage going into those areas. When movement costs vary this is used in place of Breadth First Search. Dijkstra’s Algorithm works well to find the shortest path, but it wastes time exploring in directions that aren’t promising. Best First Search explores in promising directions, but it may not find the shortest path. The A* algorithm uses both the actual distance from the start and the estimated distance to the goal.

Hardware Constraints

Hardware Restraints:

Hardware Restraints In AI Engines:

When a developer is developing a game with A.I, one of the things they must consider is the hardware constraints that machine in question the game is trying to play on. In simple terms, the game must require a certain and specific requirement in order to run smoothly without any sort of lagging, freezing or crashing. In games the software always requires the machine to do the processing; an output and input. The hardware required to play the game changes based on what type of game you are playing and how graphically intense it is, as well as how much the processor needs to process.

For example, the game Fortnite requires the following hardware:

·         Operating System: Windows 7/8/10 64-bit or Mac OSX Sierra

·         Processor Core: i3 2.4 Ghz

·         Memory: 4 GB RAM

·         Video Card: Intel HD 4000

(Russell, 2018)

As a result, a PC running Windows 10 with a i7 processor, 500SSD and 8GB RAM will have no issue running a game like Fortnite, it’s frame-rate would be capable of running a high FPS (Frame Per Second) and grant its user with a fairly-good experience of gameplay. In contrast, if the player was trying to play this game on an Intel Pentium 4 processor, with 4GB RAM, 500GB HDD and a standard Intel HD built in graphics card, then the player experience would be slow-loading, crashing and bring frustration. Typically, older PCs lack the processing power in order to run newer games.

Player Experience:

In terms of AI in games, Artificial Intelligence can either improve or break the player’s experience during gameplay. One of the obvious examples is to create more lifelike AI characters that can converse with players, compete and provide a more realistic experience. AI can also be used in somewhat unexpected ways to improve player experience. One of the most promising areas is to provide immediate help within the game itself. (Lent, 2018)

On the other hand, there can be certain factors that be ruin a player’s experience. The reason behind this is usually because of bugs or hacks in the coding. The less bugs that are within a game, the better the player experience that one will receive and having such issues like bugs (or hacks) can break the player’s immersion during gameplay it’s not logical, or the AI character behaves in a manner that is not expected. This may or not result in the player seeking support.

Algorithms, Heuristics & Hacks

Algorithms:

An algorithm is a well-defined procedure that allows a computer to solve a problem. Another way to describe an algorithm is a sequence of unambiguous instructions. In terms of games, an algorithm is a very basic set of rules that the AI character follows during gameplay in order to simulate real-world behaviour. This is applied to every type of game in some way or another, because they allow its developers to give the AI something to do either by telling how and when they should more, and what their end goals are and how they can achieve it.

Heuristics:

A heuristic is any approach to problem solving, learning, or discovery that employs a practical (though, not guaranteed) method towards reaching an immediate goal. They are used to rapidly come to a solution that is close to the best possible answer, or 'optimal solution'.  A heuristics is a "rule of thumb", an approximate solution that might work in many situations but is unlikely to work in all. Regardless, however, in some videogames these days if an NPC cannot find a can’t find a suitable solution to getting stuck or not finding a path they’ll teleport to an open space or in the vicinity of the player. This is most noticeable in games like Bethesda’s Elder Scrolls: Skyrim or Fallout 4 (Both of which use the same Game Engine), in that the NPCs might get stuck in the environment and if it cannot find a path then it will teleport in order to get around this.

Hacks:

A hack is when an AI Entity which is using information, they shouldn’t have access too; This means when the AI player suddenly does something and react in a manner that we do not expect. The AI player had set off a trigger and the AI responds to this “trigger”. A hack is a piece of code which forces an AI character to respond in a very predictable way. This is sometimes necessary to make the AI difficult or to trigger a certain event for the story of the game etc. A player character can often use this to their own advantage in-game. However, this “hack” can break story immersion and make the gameplay become somewhat unrealistic as a result.