
# Real-time Fallacy Detection


## Overview

This project aims to perform real-time fallacy detection during events like presidential debates. It uses the [Whisper](https://github.com/openai/whisper) for audio transcription.  For natural language understanding and fallacy classification you have the option to use the OpenAI ChatGPT API or a local LLM through the [text-generation-webui](https://github.com/oobabooga/text-generation-webui). I was able to run both whisper with the [IconicAI_NeuralHermes-2.5-Mistral-7B-exl2-5bpw](https://huggingface.co/IconicAI/NeuralHermes-2.5-Mistral-7B-exl2-5bpw) on a laptop with 3080TI 16GB of VRAM.

[Watch Video](https://www.youtube.com/watch?v=I9ScRL_10So)

## Features

- **Real-time Audio Transcription**: Uses OpenAI's Whisper ASR model for accurate real-time transcription locally or through API access.
- **Fallacy Detection**: Utilizes OpenAI's ChatGPT to classify and identify fallacies in real-time.
- **Overlay Display**: Provides a transparent overlay display to show both the transcription and fallacy detection results.
- **Text analysis**: using GPT-3/4 or local LLM. 


## Dependencies
- Anaconda
- PyQt5 - Overlay
- PyAudio - Audio processing
- openai-whisper - ASR
- openai - ChatGPT API
- torch with cuda, for real time transtriction 
- Have the text-generation-webui running with the API flag 

## Installation
I build the application using Anaconda with python 3.9 on windows 

1. Clone the repository:
    ```
    git clone https://github.com/latent-variable/Real_time_fallacy_detection.git
    ```
2. Navigate to the project directory:
    ```
    cd real-time-fallacy-detection
    ```
3. Create a conda environment:
    ```
    conda create --name rtfd python=3.9
    ```
4. Activate the created environment:
    ```
    conda activate rtfd
    ```
5. Install the required packages:
    ```
    pip install -r requirements.txt
    ```
6. (Optional) Install the required packages to run whisper locally:
    ```
    pip install -r requirements_local_whisper.txt
    ```
7. Installing [VB-Audio](https://vb-audio.com/Cable/), to forward audio ouput as an input device (*Optional, but I don't know how to redirect audio otherwise)

## Usage

Run the main script to start the application:
```
python main.py 
optional arguments:
  -h, --help     show this help message and exit
  --auto         Automatically get commentary
  --api_whisper  Run whisper through api instead of locally
  --api_gpt      Will use use gpt api otherwise, by defualt will use a local LLM through the text-generation-webui API
```

If you plan to use your local LLM, please have the text-generation-webui running with the --api flag

**Note**: The application routes the audio to VB-AUDIO for processing and then redirects it back to the user for playback. 

## Arguments
--use_gpt(3/4): Use this flag to toggle between  ChatGPT with or without GPT4. Default is to use local LLM.

## Display
The application will display an overlay with two sections:

- **Top Box**: Displays the fallacy classification from ChatGPT.
- **Bottom Box**: Displays the real-time transcription from the Whisper API.

Press the `Esc` key to close the application.

## Configuration
[Audio Settings]
You can configure the audio input and outsource in the `settings.ini`.
- **device_input_name = VB-Audio**  <- must have 
- **device_output_name = Headphones (2- Razer** <- replace with your own 
*Note: when the application loads if will redirect the audio back to the output device


[Local LLM Settings]
- **instruction_template = mistral-openorca** <- replace with the model specific template
*Note: this is a custom template, which you will likely note have in your text-generation-webui


Rename the `api_key.txt.template` to `api_key.txt` and add your OpenAI API key to it.

## License

This project is licensed under the MIT License.

	a/README.md		b/README.md
1		1	# Real-time Fallacy Detection
2	# Real-time Fallacy Detection	2
3
4	## Overview	3	## Overview
5		4
6	This project aims to perform real-time fallacy detection during events like presidential debates. It uses the [Whisper](https://github.com/openai/whisper) for audio transcription. For natural language understanding and fallacy classification you have the option to use the OpenAI ChatGPT API or a local LLM through the [text-generation-webui](https://github.com/oobabooga/text-generation-webui). I was able to run both whisper with the [IconicAI_NeuralHermes-2.5-Mistral-7B-exl2-5bpw](https://huggingface.co/IconicAI/NeuralHermes-2.5-Mistral-7B-exl2-5bpw) on a laptop with 3080TI 16GB of VRAM.	5	This project aims to perform real-time fallacy detection during events like presidential debates. It uses the [Whisper](https://github.com/openai/whisper) for audio transcription. For natural language understanding and fallacy classification you have the option to use the OpenAI ChatGPT API or a local LLM through the [text-generation-webui](https://github.com/oobabooga/text-generation-webui). I was able to run both whisper with the [IconicAI_NeuralHermes-2.5-Mistral-7B-exl2-5bpw](https://huggingface.co/IconicAI/NeuralHermes-2.5-Mistral-7B-exl2-5bpw) on a laptop with 3080TI 16GB of VRAM.
7	![Alt text](img/Fallacy_classification.PNG)	6
8	[Watch Video](https://www.youtube.com/watch?v=I9ScRL_10So)	7	[Watch Video](https://www.youtube.com/watch?v=I9ScRL_10So)
9		8
10	## Features	9	## Features
11		10
12	- Real-time Audio Transcription: Uses OpenAI's Whisper ASR model for accurate real-time transcription locally or through API access.	11	- Real-time Audio Transcription: Uses OpenAI's Whisper ASR model for accurate real-time transcription locally or through API access.
13	- Fallacy Detection: Utilizes OpenAI's ChatGPT to classify and identify fallacies in real-time.	12	- Fallacy Detection: Utilizes OpenAI's ChatGPT to classify and identify fallacies in real-time.
14	- Overlay Display: Provides a transparent overlay display to show both the transcription and fallacy detection results.	13	- Overlay Display: Provides a transparent overlay display to show both the transcription and fallacy detection results.
15	- Text analysis: using GPT-3/4 or local LLM.	14	- Text analysis: using GPT-3/4 or local LLM.
16		15
17		16
18	## Dependencies	17	## Dependencies
19	- Anaconda	18	- Anaconda
20	- PyQt5 - Overlay	19	- PyQt5 - Overlay
21	- PyAudio - Audio processing	20	- PyAudio - Audio processing
22	- openai-whisper - ASR	21	- openai-whisper - ASR
23	- openai - ChatGPT API	22	- openai - ChatGPT API
24	- torch with cuda, for real time transtriction	23	- torch with cuda, for real time transtriction
25	- Have the text-generation-webui running with the API flag	24	- Have the text-generation-webui running with the API flag
26		25
27	## Installation	26	## Installation
28	I build the application using Anaconda with python 3.9 on windows	27	I build the application using Anaconda with python 3.9 on windows
29		28
30	1. Clone the repository:	29	1. Clone the repository:
31	```	30	```
32	git clone https://github.com/latent-variable/Real_time_fallacy_detection.git	31	git clone https://github.com/latent-variable/Real_time_fallacy_detection.git
33	```	32	```
34	2. Navigate to the project directory:	33	2. Navigate to the project directory:
35	```	34	```
36	cd real-time-fallacy-detection	35	cd real-time-fallacy-detection
37	```	36	```
38	3. Create a conda environment:	37	3. Create a conda environment:
39	```	38	```
40	conda create --name rtfd python=3.9	39	conda create --name rtfd python=3.9
41	```	40	```
42	4. Activate the created environment:	41	4. Activate the created environment:
43	```	42	```
44	conda activate rtfd	43	conda activate rtfd
45	```	44	```
46	5. Install the required packages:	45	5. Install the required packages:
47	```	46	```
48	pip install -r requirements.txt	47	pip install -r requirements.txt
49	```	48	```
50	6. (Optional) Install the required packages to run whisper locally:	49	6. (Optional) Install the required packages to run whisper locally:
51	```	50	```
52	pip install -r requirements_local_whisper.txt	51	pip install -r requirements_local_whisper.txt
53	```	52	```
54	7. Installing [VB-Audio](https://vb-audio.com/Cable/), to forward audio ouput as an input device (*Optional, but I don't know how to redirect audio otherwise)	53	7. Installing [VB-Audio](https://vb-audio.com/Cable/), to forward audio ouput as an input device (*Optional, but I don't know how to redirect audio otherwise)
55		54
56	## Usage	55	## Usage
57		56
58	Run the main script to start the application:	57	Run the main script to start the application:
59	```	58	```
60	python main.py	59	python main.py
61	optional arguments:	60	optional arguments:
62	-h, --help show this help message and exit	61	-h, --help show this help message and exit
63	--auto Automatically get commentary	62	--auto Automatically get commentary
64	--api_whisper Run whisper through api instead of locally	63	--api_whisper Run whisper through api instead of locally
65	--api_gpt Will use use gpt api otherwise, by defualt will use a local LLM through the text-generation-webui API	64	--api_gpt Will use use gpt api otherwise, by defualt will use a local LLM through the text-generation-webui API
66	```	65	```
67		66
68	If you plan to use your local LLM, please have the text-generation-webui running with the --api flag	67	If you plan to use your local LLM, please have the text-generation-webui running with the --api flag
69		68
70	Note: The application routes the audio to VB-AUDIO for processing and then redirects it back to the user for playback.	69	Note: The application routes the audio to VB-AUDIO for processing and then redirects it back to the user for playback.
71		70
72	## Arguments	71	## Arguments
73	--use_gpt(3/4): Use this flag to toggle between ChatGPT with or without GPT4. Default is to use local LLM.	72	--use_gpt(3/4): Use this flag to toggle between ChatGPT with or without GPT4. Default is to use local LLM.
74		73
75	## Display	74	## Display
76	The application will display an overlay with two sections:	75	The application will display an overlay with two sections:
77		76
78	- Top Box: Displays the fallacy classification from ChatGPT.	77	- Top Box: Displays the fallacy classification from ChatGPT.
79	- Bottom Box: Displays the real-time transcription from the Whisper API.	78	- Bottom Box: Displays the real-time transcription from the Whisper API.
80		79
81	Press the `Esc` key to close the application.	80	Press the `Esc` key to close the application.
82		81
83	## Configuration	82	## Configuration
84	[Audio Settings]	83	[Audio Settings]
85	You can configure the audio input and outsource in the `settings.ini`.	84	You can configure the audio input and outsource in the `settings.ini`.
86	- device_input_name = VB-Audio <- must have	85	- device_input_name = VB-Audio <- must have
87	- device_output_name = Headphones (2- Razer <- replace with your own	86	- device_output_name = Headphones (2- Razer <- replace with your own
88	*Note: when the application loads if will redirect the audio back to the output device	87	*Note: when the application loads if will redirect the audio back to the output device
89	![Alt text](img/audio_selection.png)	88
90
91	[Local LLM Settings]	89	[Local LLM Settings]
92	- instruction_template = mistral-openorca <- replace with the model specific template	90	- instruction_template = mistral-openorca <- replace with the model specific template
93	*Note: this is a custom template, which you will likely note have in your text-generation-webui	91	*Note: this is a custom template, which you will likely note have in your text-generation-webui
94		92
95		93
96	Rename the `api_key.txt.template` to `api_key.txt` and add your OpenAI API key to it.	94	Rename the `api_key.txt.template` to `api_key.txt` and add your OpenAI API key to it.
97		95
98	## License	96	## License
99		97
100	This project is licensed under the MIT License.	98	This project is licensed under the MIT License.