7 | 电影数据

**电影数据分析**

在本文中，我们将使用Python语言进行电影数据分析。我们将从导入必要的库开始，然后加载电影数据集，接着进行一些基本的统计分析和可视化。

### 导入必要的库

import pandas as pdimport numpy as npimport matplotlib.pyplot as pltfrom sklearn.model_selection import train_test_split

# 下载数据集url = " />data_dir = "aclImdb"

!wget -q -O aclImdb_v1.tar.gz $url!tar xvfz aclImdb_v1.tar.gz# 加载数据集train_data = pd.read_csv(data_dir + "/train.csv")
test_data = pd.read_csv(data_dir + "/test.csv")

print(train_data.head())

# 计算平均长度avg_len_train = train_data["text"].str.len().mean()
avg_len_test = test_data["text"].str.len().mean()

print("Average length of training data:", avg_len_train)
print("Average length of testing data:", avg_len_test)

# 找到最长的评论max_len_train = train_data["text"].str.len().max()
max_len_test = test_data["text"].str.len().max()

print("Maximum length of training data:", max_len_train)
print("Maximum length of testing data:", max_len_test)

# 统计评论长度分布train_len_dist = train_data["text"].str.len().value_counts()
test_len_dist = test_data["text"].str.len().value_counts()

plt.figure(figsize=(10,6))
plt.bar(train_len_dist.index, train_len_dist.values)
plt.title("Training Data Length Distribution")
plt.xlabel("Length")
plt.ylabel("Frequency")
plt.show()

plt.figure(figsize=(10,6))
plt.bar(test_len_dist.index, test_len_dist.values)
plt.title("Testing Data Length Distribution")
plt.xlabel("Length")
plt.ylabel("Frequency")
plt.show()

# 分割训练数据集train_text, val_text, train_labels, val_labels = train_test_split(train_data["text"], train_data["label"], test_size=0.2, random_state=42)

print("Training text shape:", train_text.shape)
print("Validation text shape:", val_text.shape)
print("Training labels shape:", train_labels.shape)
print("Validation labels shape:", val_labels.shape)