使用torch7进行文本分类（一） - 推酷
使用torch7进行文本分类（一）
博客荒废了很久准备重新开始写起来。
最近一直在忙毕设的事情准备用深度学习的方法进行微博情感分析，在我们的研究中，将使用5分类的方法来将微博进行分类。之前纠结深度学习工具的选择，先后在theano，deeplearningtoolbox，torch和deeplarining4j之间纠结了很久，选来选去最终还是选了torch7。具体原因先按下不表，过程很纠结就是了。
torch7除了标准的nn包之外，还提供了dp包来进行深度学习，本文就利用了dp包进行编程。处理了固定长度（12词）的文本分类问题。不同长度的代码还在编写中。
数据准备方面，首先是用了word2vec工具将分词后的文本都学习成了embeded vector每个词向量长度100，将词长度为12的句子挑选出来进行采样，制作成训练集、验证集合测试集，相当于每个句子是一个1200维的向量。五个文件，分别代表5个分类。
在读取文件的时候，我把1&1200维的向量reshape成了12*100维的向量。prepareData.lua
for _,dataset_name in ipairs({&train&,&valid&,&test&}) do
classes=nil
path_prefix=os.getenv('HOME')..&/data/weibo/&
th_output_prefix=os.getenv('HOME')..&/workspace/torch7/&
path_surfix=&.txt&
for _,index in ipairs({0,1,2,3,4}) do
classes_n={}
file=io.open(path_prefix..dataset_name..index..path_surfix,'r')
for line in file:lines() do
line_vector={}
for element in string.gmatch(line,&%S+&) do
table.insert(line_vector,element)
table.insert(data_n,line_vector)
data_tensor_n=torch.Tensor(data_n)
data_tensor_n=data_tensor_n:resize(data_tensor_n:size(1),data_tensor_n:size(2)/100,100)
classes_tensor_n=torch.Tensor(data_tensor_n:size(1)):fill(index)
print(data_tensor_n:size())
print(classes_tensor_n:size())
datas=datas and torch.cat(datas,data_tensor_n,1) or data_tensor_n
classes=classes and torch.cat(classes,classes_tensor_n,1) or classes_tensor_n
classes=classes:int()
print(datas:size())
print(classes:size())
data_object={datas,classes}
torch.save(th_output_prefix..dataset_name..'.th7',data_object)
制作3个数据文件，分别取名为train.th7,valid.th7和test.th7。
datasource编写
利用mnist的datasource改的。注意，我们需要的输入是一个SequenceView，也就是可以用来做1维卷积的View。SequenceView中的bwc分别代表“batch大小”，“句子长度”和“embedVector的大小”
local Weibo, DataSource = torch.class(&dp.Weibo&, &dp.DataSource&)
Weibo.isMnist = true
Weibo._name = 'weibo'
Weibo._text_axes = 'bwc'
Weibo._classes = {0, 1, 2, 3, 4, }
function Weibo:__init(config)
config = config or {}
assert(torch.type(config) == 'table' and not config[1],
&Constructor requires key-value arguments&)
local args, load_all, input_preprocess, target_preprocess
args, self._valid_ratio, self._train_file, self._test_file, self._valid_file,
self._data_path, self._scale, self._binarize, self._shuffle, load_all, input_preprocess,
target_preprocess
= xlua.unpack(
'Handwritten digit classification problem.' ..
'Note: Train and valid sets are already shuffled.',
{arg='valid_ratio', type='number', default=1/6,
help='proportion of training set to use for cross-validation.'},
{arg='train_file', type='string', default='train.th7',
help='name of training file'},
{arg='valid_file', type='string', default='valid.th7',
help='name of valid file'},
{arg='test_file', type='string', default='test.th7',
help='name of test file'},
{arg='data_path', type='string', default=dp.DATA_DIR,
help='path to data repository'},
{arg='scale', type='table',
help='bounds to scale the values between. [Default={0,1}]'},
{arg='binarize', type='boolean',
help='binarize the inputs (0s and 1s)', default=false},
{arg='shuffle', type='boolean',
help='shuffle different sets', default=false},
{arg='load_all', type='boolean',
help='Load all datasets : train, valid, test.', default=true},
{arg='input_preprocess', type='table | dp.Preprocess',
help='to be performed on set inputs, measuring statistics ' ..
'(fitting) on the train_set only, and reusing these to ' ..
'preprocess the valid_set and test_set.'},
{arg='target_preprocess', type='table | dp.Preprocess',
help='to be performed on set targets, measuring statistics ' ..
'(fitting) on the train_set only, and reusing these to ' ..
'preprocess the valid_set and test_set.'}
self:loadTrain()
self:loadValid()
self:loadTest()
DataSource.__init(self, {
train_set=self:trainSet(), valid_set=self:validSet(),
test_set=self:testSet(), input_preprocess=input_preprocess,
target_preprocess=target_preprocess
function Weibo:loadTrain()
local train_data = self:loadData(self._train_file)
self:setTrainSet(
self:createDataSet(train_data[1], train_data[2], 'train')
return self:trainSet()
function Weibo:loadValid()
local valid_data = self:loadData(self._valid_file)
self:setValidSet(
self:createDataSet(valid_data[1], valid_data[2], 'valid')
return self:validSet()
function Weibo:loadTest()
local test_data = self:loadData(self._test_file)
self:setTestSet(
self:createDataSet(test_data[1], test_data[2], 'test')
return self:testSet()
function Weibo:createDataSet(inputs, targets, which_set)
if self._shuffle then
local indices = torch.randperm(inputs:size(1)):long()
inputs = inputs:index(1, indices)
targets = targets:index(1, indices)
if self._binarize then
DataSource.binarize(inputs, 128)
-- class 0 will have index 1, class 1 index 2, and so on.
targets:add(1)
-- construct inputs and targets dp.Views
local input_v, target_v = dp.SequenceView(), dp.ClassView()
input_v:forward(self._text_axes, inputs)
target_v:forward('b', targets)
target_v:setClasses(self._classes)
-- construct dataset
dataset= dp.DataSet{inputs=input_v,targets=target_v,which_set=which_set}
--print(dataset)
return dataset
function Weibo:loadData(file_name)
local path=&../&..file_name
print(file_name)
-- backwards compatible with old binary format
local status, data = pcall(function() return torch.load(path) end)
if not status then
return torch.load(path, &binary&)
return data
实验代码编写
使用cnn的方式处理，分为三层，第一层是一个一维卷积，第二层和第三层都是传统的神经网络写法。
require 'dp'
require 'weiboSource'
--[[hyperparameters]]--
nHidden = 100, --number of hidden units
learningRate = 0.1, --training learning rate
momentum = 0.9, --momentum factor to use for training
maxOutNorm = 1, --maximum norm allowed for output neuron weights
batchSize = 128, --number of examples per mini-batch
maxTries = 100, --maximum number of epochs without reduction in validation error.
maxEpoch = 1000, --maximum number of epochs of training
cuda =false,
useDevice =1,
inputEmbeddingSize =100,
outputEmbeddingSize=100,
convOutputSize=50,
convKernelSize=2,
convKernelStride=1,
convPoolSize=2,
convPoolStride=2,
contextSize=4,
decayPoint=100 ,--epoch at which learning rate is decayed
decayFactor=0.1, --'factory by which learning rate is decayed at decay point'
local datasource=dp.Weibo()
inputModel = dp.Convolution1D{
input_size = opt.inputEmbeddingSize,
output_size = opt.convOutputSize,
kernel_size = opt.convKernelSize,
kernel_stride = opt.convKernelStride,
pool_size = opt.convPoolSize,
pool_stride = opt.convPoolStride,
transfer = nn.Tanh(),
dropout = opt.dropout and nn.Dropout() or nil,
acc_update = opt.accUpdate
local nOutputFrame = inputModel:outputSize(opt.contextSize, 'bwc')
dp.vprint(not opt.silent, &Convolution has &..nOutputFrame..& output Frames&)
inputSize = nOutputFrame*opt.convOutputSize
--print(hiddenModel)
softmax = dp.Neural{
input_size = opt.outputEmbeddingSize,
output_size = table.length(datasource:classes()),
transfer = nn.LogSoftMax(),
dropout = opt.dropout and nn.Dropout() or nil,
acc_update = opt.accUpdate
mlp = dp.Sequential{
models = {
inputModel,
dp.Neural{
input_size = inputSize,
output_size = opt.outputEmbeddingSize,
transfer = nn.Tanh(),
dropout = opt.dropout and nn.Dropout() or nil,
acc_update = opt.accUpdate
--[[Propagators]]--
train = dp.Optimizer{
loss = opt.softmaxtree and dp.TreeNLL() or dp.NLL(),
visitor = {
learning_rate = opt.learningRate,
observer = dp.LearningRateSchedule{
schedule = {[opt.decayPoint]=opt.learningRate*opt.decayFactor}
dp.MaxNorm{max_out_norm=opt.maxOutNorm, period=opt.maxNormPeriod}
feedback = dp.Perplexity(),
sampler = dp.Sampler{ --shuffle sample takes too much mem
epoch_size = opt.trainEpochSize, batch_size = opt.batchSize
progress = opt.progress
valid = dp.Evaluator{
loss = opt.softmaxtree and dp.TreeNLL() or dp.NLL(),
feedback = dp.Perplexity(),
sampler = dp.Sampler{
epoch_size = opt.validEpochSize,
batch_size = opt.softmaxtree and 1024 or opt.batchSize
progress = opt.progress
tester = dp.Evaluator{
loss = opt.softmaxtree and dp.TreeNLL() or dp.NLL(),
feedback = dp.Perplexity(),
sampler = dp.Sampler{batch_size = opt.softmaxtree and 1024 or opt.batchSize}
--[[Experiment]]--
xp = dp.Experiment{
model = mlp,
optimizer = train,
validator = valid,
tester = tester,
observer = (not opt.trainOnly) and {
dp.FileLogger(),
dp.EarlyStopper{max_epochs = opt.maxTries}
random_seed = os.time(),
max_epoch = opt.maxEpoch
--[[GPU or CPU]]--
if opt.cuda then
require 'cutorch'
require 'cunn'
if opt.softmaxtree or opt.softmaxforest then
require 'cunnx'
cutorch.setDevice(opt.useDevice)
print&dp.Models :&
print(mlp)
print&nn.Modules :&
trainset=datasource:trainSet():sub(1,32)
print(mlp:toModule(datasource:trainSet():sub(1,32)))
xp:run(datasource)
实验结果在测试集上5分类达到了70%+，令我感到十分意外，真是意外之喜
已发表评论数()
请填写推刊名
描述不能大于100个字符!
权限设置： 公开
仅自己可见
正文不准确
标题不准确
排版有问题
主题不准确
没有分页内容
图片无法显示
视频无法显示
与原文不一致Torch7深度学习教程（三）_词汇网
Torch7深度学习教程（三）
责任编辑：词汇网 发表时间： 14:33:43
函数的使用
这是函数的定义方式，声明的关键字+定义的函数名+形参的名字，在此博主返回两个值，具体的函数功能在后面再说
这是初始化一个5x2的矩阵，并且初值都为1。这里有多了一种初始化矩阵的方法。
这是先声明一个2x5的矩阵，然后再调用fill（）方法其值全部初始化为4。
将a，b矩阵输入到addTensors函数里面，注意这里是实参，前面定义的a，b是形参，这个有点c基础的应该都分的清楚吧。打印结果就是返回a矩阵和axb的矩阵。
这样子写更像matlab，返回的a和axb分别按顺序赋值给c，d。CUDA的Tensor 抱歉了，我的电脑不是NVIDIA的显卡，这里以后再补上吧，看不到运行结果 这里就是多调用一下cuda（）函数，然后计算axb矩阵的乘机运算就可以了。开头的require ‘cutorch’就是导入cuda运算包，类似于c语言里面的include。下一章，开始进入主题――神经网络
上一集:没有了 下一集:
相关文章：&&&&&&&&&&&&&&
最新添加资讯
24小时热门资讯
附近好友搜索