Année
2024
Rôle
Pipeline implementation and comparative benchmarking
Technologies
- Apache Hadoop
- Apache Spark
- Naive Bayes
- SVM
- Sentiment140
Distributed computing
Large-Scale Text Sentiment Classification
Sentiment classification of 1.6M tweets (Sentiment140) with Naive Bayes and SVM, run on Apache Hadoop and Apache Spark.
- This project explored text mining, sentiment analysis, and parallel/distributed computing with Apache Hadoop and Apache Spark.
- The system was implemented on the Sentiment140 dataset (1.6M tweets) using Naive Bayes and SVM.
- Results showed that Naive Bayes achieved 75% accuracy with faster processing speed, while SVM reached higher accuracy (80%–82%) but required longer training time on Hadoop.
- Spark outperformed Hadoop in execution time and scalability for large-scale data.
- Dataset
- 1.6M
- tweets, Sentiment140
- SVM accuracy
- 80–82%
- longer training time on Hadoop
- Naive Bayes accuracy
- 75%
- faster processing
