博客访问： 3876614
博文数量： 536
博客积分： 13065
博客等级： 上将
技术积分： 9056
注册时间：
IT168企业级官微
微信号：IT168qiye
系统架构师大会
微信号：SACC2013
分类： LINUX
&std::basic_string&unsigned char,="" std::basic_string除非明确说明，本文内容仅针对x86/x86_64的Linux开发环境，有朋友说baidu不到，开个贴记录一下（加粗字体是关键词）：
用“-Wl,-Bstatic”指定链接静态库，使用“-Wl,-Bdynamic”指定链接共享库，使用示例：
-Wl,-Bstatic -lmysqlclient_r -lssl -lcrypto -Wl,-Bdynamic -lrt -Wl,-Bdynamic -pthread -Wl,-Bstatic -lgtest
（"-Wl"表示是传递给链接器ld的参数，而不是编译器gcc/g++的参数。）
1) 下面是因为没有指定链接参数-lz（/usr/lib/libz.so，/usr/lib/libz.a ）
/usr/local/mysql/lib/mysql/libmysqlclient.a(my_compress.c.o): In function `my_uncompress':
/home/software/mysql-5.5.24/mysys/my_compress.c:122: undefined reference to `uncompress'
/usr/local/mysql/lib/mysql/libmysqlclient.a(my_compress.c.o): In function `my_compress_alloc':
/home/software/mysql-5.5.24/mysys/my_compress.c:71: undefined reference to `compress'
2) 下面是因为没有指定编译链接参数-pthread（注意不仅仅是-lpthraed）
/usr/local/mysql/lib/mysql/libmysqlclient.a(charset.c.o): In function `get_charset_name':
/home/zhangsan/mysql-5.5.24/mysys/charset.c:533: undefined reference to `pthread_once'
3) 下面这个是因为没有指定链接参数-lrt
/usr/local/thirdparty/curl/lib/libcurl.a(libcurl_la-timeval.o): In function `curlx_tvnow':
timeval.c:(.text+0xe9): undefined reference to `clock_gettime'
4) 下面这个是因为没有指定链接参数-ldl
/usr/local/thirdparty/openssl/lib/libcrypto.a(dso_dlfcn.o): In function `dlfcn_globallookup':
dso_dlfcn.c:(.text+0x4c): undefined reference to `dlopen'
dso_dlfcn.c:(.text+0x62): undefined reference to `dlsym'
dso_dlfcn.c:(.text+0x6c): undefined reference to `dlclose'
5) 下面这个是因为指定了链接参数-static，它的存在，要求链接的必须是静态库，而不能是共享库
ld: attempted static link of dynamic object
如果是以-L加-l方式指定，则目录下必须有.a文件存在，否则会报-l的库文件找不到：ld: cannot find -lACE
6) GCC编译遇到如下的错误，可能是因为在编译时没有指定-fPIC，记住：-fPIC即是编译参数，也是链接参数
relocation R_x86_64_32S against `vtable for CMyClass` can not be used when making a shared object
7) 下面的错误表示gcc编译时需要定义宏__STDC_FORMAT_MACROS，并且必须包含头文件inttypes.h
test.cpp:35: error: expected `)' before 'PRIu64'
8) 下面是因为在x86机器（32位）上编译没有指定编译参数-march=pentium4
../../src/common/libmooon.a(logger.o): In function `atomic_dec_and_test':
../../include/mooon/sys/atomic_gcc.h:103: undefined reference to `__sync_sub_and_fetch_4'
9) 下列错误可能是因为多了个“}”
error: expected declaration before '}' token
10) 下列错误可能是因为少了个“}”
error: expected `}' at end of input
11) 下面这个错误是编译一个共享库时，该共享库依赖的一静态库编译时没有加“-fPIC”参数，解决方法为带“-fPIC”重新编译被依赖的静态库
relocation R_X86_64_32 against `a local symbol' can not be used when ma recompile with -fPIC
12) 下面这个错误，是因为头文件中使用“_syscall0(pid_t, gettid)”不当引起的
./test.o: In function `gettid()':
./test.h:17: multiple definition of `gettid()'
正确的用法是使用"inline"或"static inline"修饰一下：
inline _syscall0(pid_t, gettid)
static inline _syscall0(pid_t, gettid)
当然也可以这样：
在.h头文件中：extern "C" pid_t gettid(void);
在.cpp文件中：_syscall0(pid_t, gettid)
_syscall0是一个宏，定义一个函数的实现。
13) 下列编译告警是因为一个static类型的函数未被使用
my.cpp:364: warning: 'int my_function(const cgicc::Cgicc&, const std::string&)' defined but not used
只需使用“__attribute__((unused))”修饰函数的声明即可：
static int __attribute__((unused)) my_function(const cgicc::Cgicc&, const std::string&);
14) 执行thrift的configure时遇到如下的错误（thrift-0.9.0和thrift-0.9.1遇到过）：
checking for setsockopt in -lsocket... no
checking for BN_init in -lcrypto... no
configure: error: "Error: libcrypto required."
原因可能是因为编译安装openssl时指定了--prefix，比如--prefix=/usr/local/thirdparty/openssl，可这样解决：
不指定thrift的configure的--with-openssl=/usr/local/thirdparty/openssl，改为：
CPPFLAGS="-I/usr/local/thirdparty/openssl/include" LDFLAGS="-ldl -L/usr/local/thirdparty/openssl/lib"
替代它就OK了。
15) 下面这个编译错误（表现为g++进入死循环），可能是由于缺少右大括号“}”导致的，比如定义名字空间时少了“}”：
/usr/include/c++/4.1.2/tr1/type_traits:408: error: 'size_t' is not a member of 'db_proxy::std'
/usr/include/c++/4.1.2/tr1/type_traits:408: error: 'size_t' is not a member of 'db_proxy::std'
/usr/include/c++/4.1.2/tr1/mu_iterate.h:49: error: 'std::tr1' has not been declared
/usr/include/c++/4.1.2/tr1/mu_iterate.h:49: error: 'std::tr1' has not been declared
/usr/include/c++/4.1.2/tr1/bind_iterate.h:78: error: 'std::tr1' has not been declared
/usr/include/c++/4.1.2/tr1/bind_iterate.h:78: error: 'std::tr1' has not been declared
/usr/include/c++/4.1.2/tr1/bind_iterate.h:78: error: 'std::tr1' has not been declared
16) protoc编译错误，下面错误是因为没有在.proto文件所在目录下执行：
/tmp/test.proto: File does not reside within any path specified using --proto_path (or -I). &You must specify a --proto_path which encompasses this file. &Note that the proto_path must be an exact prefix of the .proto file names -- protoc is too dumb to figure out when two paths (e.g. absolute and relative) are equivalent (it's harder than you think).
解决办法有两个：一是在.proto文件所在目录下执行protoc，二是为protoc指定参数--proto_path，参数值为.proto文件所在目录。
17) 下面这个编译错误，可能是因为在全局域内调用一个类对象的成员函数，全局域内是不能直接执行函的：
error: expected constructor, destructor, or type conversion before '-&' token
18) 下面这个错误是因为没有链接OpenSSL的libcrypto库，或者使用了静态库，而顺序不对：
undefined symbol: EVP_enc_null
19) 下列是链接错误，不是编译错误，加上“-pthread”即可，注意不是“-lpthread”：
/usr/local/mysql/lib/mysql/libmysqlclient.a(charset.c.o): In function `get_charset_name':
/home/software/mysql-5.5.24/mysys/charset.c:533: undefined reference to `pthread_once'
/usr/local/mysql/lib/mysql/libmysqlclient.a(charset.c.o): In function `get_charset_number':
20) 依赖OpenSSL（假设安装在/usr/local/thirdparty/openssl-1.0.2a）时，如果
./configure --prefix=/usr/local/thirdparty/libssh2-1.6.0 --with-libssl-prefix=/usr/local/thirdparty/openssl-1.0.2a
时遇到如下错误：
configure: error: No crypto library found!
Try --with-libssl-prefix=PATH
&or --with-libgcrypt-prefix=PATH
&or --with-wincng on Windows
可以如下方法解决：
./configure --prefix=/usr/local/thirdparty/libssh2-1.6.0 --with-openssl CPPFLAGS="-I/usr/local/thirdparty/openssl-1.0.2a/include" LDFLAGS="-L/usr/local/thirdparty/openssl-1.0.2a/lib"
21) 错误“error: expected primary-expression before ‘;’ token”可能是因为如下原因：
UserInfoInternal* user_info_internal = new UserInfoI
delete user_ // 这里应当是user_info_internal
22) 下面这个编译错误的原因是定义字符串宏时没有加双引引号：
example.cpp:563:16: error: too many decimal points in number
如定义成了：#define IP1 127.0.0.1，改成#define IP1 "127.0.0.1"后问题即解决。
23) 下面这个编译错误是因为g++命令参数没写对，多了个“-”
g++: -E or -x required when input is from standard input
CPPFLAGS=-pthread -
g++ -g -o $@ $^ $(CPPFLAGS) $(LDFLAGS)
24) libjson_7.6.1编译错误：
makefile:180: Extraneous text after `else' directive
makefile:183: *** only one `else' per conditional. &Stop.
解决办法：
找到makefile文件的第180行，将“else ifeq ($(BUILD_TYPE), debug)”，改成两行内嵌方式：
# BUILD_TYPE specific settings
ifeq ($(BUILD_TYPE), small)
& & CXXFLAGS & & = $(cxxflags_small)
& & ifeq ($(BUILD_TYPE), debug) # 不能和else同一行，否则Makefile语法错误，不支持else ifeq
& & & & CXXFLAGS & & = $(cxxflags_debug)
& & & & libname & & := $(libname_debug)
& & & & CXXFLAGS & &?= $(cxxflags_default)
可以通过设置环境变量来设置BUILD_TYPE，如：export BUILD_TYPE=debug
也可以通过环境变量来设置make install时的安装目录，如：export prefix=/usr/local/libjson
相关小知识：
在Makefile文件中，prefix=/usr和prefix?=/usr，是有区别的，前者赋值不能通过环境变量覆盖，后者则可以使用环境变量的值覆盖。
另外，请将第271行删除：
cp -rv $(srcdir)/Dependencies/ $(include_path)/$(libname_hdr)/$(srcdir)
cp -rv _internal/Dependencies/ $(include_path)/$(libname_hdr)/$(srcdir)
还有第258行前插入如一条命令：
mkdir -p $(inst_path)
否则“cp -f ./$(lib_target) $(inst_path)”，lib将成库文件名。
25） 编译gcc时，如果遇到下面这个错误，这是因为运行时找不到mpc、mpfr和gmp的so文件：
checking for x86_64-unknown-linux-gnu-nm... /data/gcc-4.8.2_src/host-x86_64-unknown-linux-gnu/gcc/nm
checking for x86_64-unknown-linux-gnu-ranlib... ranlib
checking for x86_64-unknown-linux-gnu-strip... strip
checking whether ln -s works... yes
checking for x86_64-unknown-linux-gnu-gcc... /data/gcc-4.8.2_src/host-x86_64-unknown-linux-gnu/gcc/xgcc -B/data/gcc-4.8.2_src/host-x86_64-unknown-linux-gnu/gcc/ -B/data/gcc-4.8.2/x86_64-unknown-linux-gnu/bin/ -B/data/gcc-4.8.2/x86_64-unknown-linux-gnu/lib/ -isystem /data/gcc-4.8.2/x86_64-unknown-linux-gnu/include -isystem /data/gcc-4.8.2/x86_64-unknown-linux-gnu/sys-include &&
checking for suffix of object files... configure: error: in `/data/gcc-4.8.2_src/x86_64-unknown-linux-gnu/libgcc':
configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details.
make[2]: *** [configure-stage1-target-libgcc] 错误 1
所以只需要如下操作下即可：
export LD_LIBRARY_PATH=/usr/local/mpc/lib:/usr/local/mpfr/lib:/usr/local/gmp/lib:$LD_LIBRARY_PATH
注：gcc-4.8.2依赖mpc、mpfr和gmp：
./configure --prefix=/usr/local/gcc-4.8.2 --with-mpc=/usr/local/mpc-1.0.3 --with-mpfr=/usr/local/mpfr-3.1.3 --with-gmp=/usr/local/gmp-6.0.0
而mpc又依赖于：mpfr和gmp：
./configure --prefix=/usr/local/mpc-1.0.3 --with-mpfr=/usr/local/mpfr-3.1.3 --with-gmp=/usr/local/gmp-6.0.0
26） 编译gcc时，如果遇到下面这个错误：
fatal error: gnu/stubs-32.h: No such file or directory
这是因为在x86_64上，默认会编译出32位和64位两个版本。这样编译32位时，需要机器上有32位的libc头文件和库文件，但一些机器上可能没有，比如没有/lib目录，只有/lib64目录，这表示不支持32位的libc。为解决这个问题，可以禁止编译32位版本，在configure时带上参数--disable-multilib，或者安装32位版本的glibc。
27）某次编译遇到如下这样一个链接错误：
redis_dbi.cpp:224: undefined reference to `sdscatlen(char*, void const*, unsigned long)'
按常理，这个错误要么是没有指定相应的库，要么是静态库间的顺序问题。
但经过检查，这两个原因，而是因为gcc和g++混用原因：
1. 库libhiredis.a和libhiredis.so是由gcc编译出来的
2. 而调用它的代码是由g++编译的，因此导致了此问题。
问题的解决办法有两个：
1. 修改sdscatlen所在的.h文件，将代码使用
#ifdef __cplusplus
extern "C" {
修饰起来，或者直接使用“extern C”修饰函数sdscatlen。
2. 不修改redis的代码，在引用sds.h时加上“extern "C" {”：
extern "C" {
#include "sds.h"
上面两个办法均可，当然也可以考虑改用g++编译redis，不过可能会遇到很多错误。
redis对外供外部直接使用的头文件hiredis.h已使用了extern "C" {，所以不存在问题，只有当跳过hiredis.h，去使用一些内部头文件时需要注意一下。
28）x.hpp:27: error: expected identifier before string constant
x.hpp:27: error: expected `}' before string constant
x.hpp:27: error: expected unqualified-id before string constant
这个错误，可能是存在和枚举等同名的字符串宏，比如存在下面的宏定义：
enum DBTYPE
& & UNDEFINE = 0,
& & MYSQL_DB = 1,
& & ORACLE_DB = 2
而另一.h文件中定义了宏：
#define MYSQL_DB "mysql"
29) 下面这个错误是因为类成员函数的声明和定义的返回值不相同
test.cpp:201:6: 错误：‘bool foo(const string&, const string&, const string&, const string&, const string&, const HBaseRowValue&)’的原型不匹配类‘CTest’中的任何一个
&bool CHBaseOper::foo(const std::string& tablename, const std::string& rowkey, const std::string& familyname, const std::string& columnname, const std::string& columnvalue, const HBaseRowValue& row_values)
In file included from test.cpp:8:0:
test.h:58:6: 错误：备选为：int foo(const string&, const string&, const string&, const string&, const string&, const HBaseRowValue&)
& int foo(const std::string& tablename, const std::string& rowkey, const std::string& familyname, const std::string& columnname, const std::string& columnvalue, const HBaseRowValue& row_values);
messenger.cpp:5: error: expected unqualified-id before ':' token
该编译错误原因是：
CMessager:CMessager()
即少了个冒号：
CMessager::CMessager()
31) unable to find a register to spill in class ‘SIREG’
编译时如果遇到这个错误，表示遇到一个gcc的bug，最简单的办法是去掉编译参数中的-O3等优化项，然后再试可能就成功了，也可以考虑指定-fno-schedule-insns。
32）像下面这样一大堆乱七八糟的错误，可以考虑是否为少了“}”等引起的
/usr/include/c++/4.8.2/bits/stl_list.h:131:15: 错误：‘bidirectional_iterator_tag’不是命名空间‘gongyi::std’中的一个类型名
& & & &typedef std::bidirectional_iterator_tag & &iterator_
& & & & & & & &^
/usr/include/c++/4.8.2/bits/stl_list.h: 在成员函数‘_Tp* gongyi::std::_List_iterator&_tp&::operator-&() const’中:
/usr/include/c++/4.8.2/bits/stl_list.h:150:16: 错误：‘__addressof’不是‘gongyi::std’的成员
& & & &{ return std::__addressof(static_cast&_Node*&(_M_node)-&_M_data); }&
namespace A { namespace B {
& & 。。。。。。
// 下面少了一个“}”
} // namespace A { namespace B {
33）crc32’ cannot be used as a function
uint32_t crc32 = crc32(0L, (const unsigned char*)crc32_str.data(), crc32_str.size());
错误是因为函数名和变量名相同了，可以改成如下解决：
uint32_t crc32 = ::crc32(0L, (const unsigned char*)crc32_str.data(), crc32_str.size());
/data/X/mooon/tools/hbase_stress.cpp: In function 'void test_write(const std::map&std::basic_string, std::basic_string &&)':
/data/X/mooon/tools/hbase_stress.cpp:78:117: error: invalid conversion from 'void (*)(uint8_t, const string&, uint16_t) {aka void (*)(unsigned char, const std::basic_string&, short unsigned int)}' to 'mooon::sys::FunctionWith3Parameter&unsigned char, std::basic_string, short unsigned int&::FunctionPtr {aka void (*)(unsigned char, std::basic_string, short unsigned int)}' [-fpermissive]
& & & & & & &stress_threads[i] = new mooon::sys::CThreadEngine(mooon::sys::bind(write_thread, i, hbase_ip, hbase_port)); & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & & ^
In file included from /data/X/mooon/tools/hbase_stress.cpp:24:0:
/data/X/mooon/tools/../include/mooon/sys/thread_engine.h:777:16: error: & initializing argument 1 of 'mooon::sys::Functor mooon::sys::bind(typename mooon::sys::FunctionWith3Parameter&Parameter1Type, Parameter2Type, Parameter3Type&::FunctionPtr, Parameter1Type, Parameter2Type, Parameter3Type) [with Parameter1Type = Parameter2Type = std::basic_ Parameter3Type = typename mooon::sys::FunctionWith3Parameter&Parameter1Type, Parameter2Type, Parameter3Type&::FunctionPtr = void (*)(unsigned char, std::basic_string, short unsigned int)]' [-fpermissive]
&inline Functor bind(typename FunctionWith3Parameter&Parameter1Type, Parameter2Type, Parameter3Type&::FunctionPtr function_ptr, Parameter1Type parameter1, Parameter2Type parameter2, Parameter3Type parameter3)
上面这个错误的意思是第一个参数的类型为
void (*)(unsigned char, std::basic_string, short unsigned int)
但传入的类型为
void (*)(unsigned char, const std::basic_string&, short unsigned int)
从上面的对比可以看出，要求函数的第二个参数为std::string类型，而不是const std::string&类型。
35) conflicting declaration
has a previous declaration as
这个错误可能是因为头文件没有#ifndef，导致引入多次。
36） warning: cannot pass objects of non-POD type ‘const struct std::basic_string&char, std::char_traits, std::allocator &’ through ‘...’; call will abort at runtime
warning: format ‘%s’ expects type ‘char*’, but argument 3 has type ‘int
这个错误是因为格式化%s参数的值为std::string值，这个在运行时可能会触发SIGILL（Illegal instruction），例如：
。。。 。。。
const std::string& key = mooon::utils::CStringUtils::format_string("%s_%s", type.c_str(), name);
&/std::basic_string
阅读(13361) | 评论(4) | 转发(0) |
相关热门文章
给主人留下些什么吧！~~
\'__curl_rule_01__\'&is&negative这个编译错误，是因为在64位平台上编译64位程序，但curl库是32位方式编译的。
35)&conflicting&declarationhas&a&previous&declaration&as这个错误可能是因为头文件没有#ifndef，导致引入多次。
编译openssl：./config&--prefix=/usr/local/openssl-1.1.0b&shared&threads
-fschedule-insnsGCC编译参数，重排序指令。-O2或-O3涵盖了-fschedule-insns。相关-fschedule-insns2、-fsched-spec-load、-fsched-spec-load-dangerous和-fsched-pressure等。
请登录后评论。Protobuf移植到AIX平台_百度文库
两大类热门资源免费畅读
续费一年阅读会员，立省24元！
Protobuf移植到AIX平台
&&介绍一个例子,如何把linux下的protobuf移植到IBM的AIX平台。
阅读已结束，下载文档到电脑
想免费下载本文？
定制HR最喜欢的简历
下载文档到电脑，方便使用
还剩6页未读，继续阅读
定制HR最喜欢的简历
你可能喜欢安全检查中...
请打开浏览器的javascript，然后刷新浏览器
< 浏览器安全检查中...
还剩 5 秒&/ protobuffertest
项目语言:C++
权限:read-only(如需更高权限请先加入项目)
protobuffertest/
Index: command_line_interface.cc
===================================================================
--- command_line_interface.cc (revision 0)
+++ command_line_interface.cc (revision 2)
@@ -0,0 +1,1603 @@
+// Protocol Buffers - Google's data interchange format
+// Copyright 2008 Google Inc.
All rights reserved.
+// /p/protobuf/
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are
* Redistributions of source code must retain the above copyright
+// notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
+// copyright notice, this list of conditions and the following disclaimer
+// in the documentation and/or other materials provided with the
+// distribution.
* Neither the name of Google Inc. nor the names of its
+// contributors may be used to endorse or promote products derived from
+// this software without specific prior written permission.
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+// &AS IS& AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+// Author:
(Kenton Varda)
Based on original Protocol Buffers design by
Sanjay Ghemawat, Jeff Dean, and others.
+#include &google/protobuf/compiler/command_line_interface.h&
+#include &stdio.h&
+#include &sys/types.h&
+#include &sys/stat.h&
+#include &fcntl.h&
+#ifdef _MSC_VER
+#include &io.h&
+#include &direct.h&
+#include &unistd.h&
+#include &errno.h&
+#include &iostream&
+#include &ctype.h&
+#include &google/protobuf/stubs/hash.h&
+#include &memory&
+#include &google/protobuf/stubs/common.h&
+#include &google/protobuf/stubs/stringprintf.h&
+#include &google/protobuf/compiler/importer.h&
+#include &google/protobuf/compiler/code_generator.h&
+#include &google/protobuf/compiler/plugin.pb.h&
+#include &google/protobuf/compiler/subprocess.h&
+#include &google/protobuf/compiler/zip_writer.h&
+#include &google/protobuf/descriptor.h&
+#include &google/protobuf/text_format.h&
+#include &google/protobuf/dynamic_message.h&
+#include &google/protobuf/io/coded_stream.h&
+#include &google/protobuf/io/zero_copy_stream_impl.h&
+#include &google/protobuf/io/printer.h&
+#include &google/protobuf/stubs/strutil.h&
+#include &google/protobuf/stubs/substitute.h&
+#include &google/protobuf/stubs/map_util.h&
+#include &google/protobuf/stubs/stl_util.h&
+namespace google {
+namespace protobuf {
+namespace compiler {
+#if defined(_WIN32)
+#define mkdir(name, mode) mkdir(name)
+#ifndef W_OK
+#define W_OK 02
// not defined by MSVC for whatever reason
+#ifndef F_OK
+#define F_OK 00
// not defined by MSVC for whatever reason
+#ifndef STDIN_FILENO
+#define STDIN_FILENO 0
+#ifndef STDOUT_FILENO
+#define STDOUT_FILENO 1
+#ifndef O_BINARY
+#ifdef _O_BINARY
+#define O_BINARY _O_BINARY
+#define O_BINARY 0
// If this isn't defined, the platform doesn't need it.
+namespace {
+#if defined(_WIN32) && !defined(__CYGWIN__)
+static const char* kPathSeparator = &;&;
+static const char* kPathSeparator = &:&;
+// Returns true if the text looks like a Windows-style absolute path, starting
+// with a drive letter.
TODO(kenton):
Share this with
+// copy in importer.cc?
+static bool IsWindowsAbsolutePath(const string& text) {
+#if defined(_WIN32) || defined(__CYGWIN__)
return text.size() &= 3 && text[1] == ':' &&
isalpha(text[0]) &&
(text[2] == '/' || text[2] == '\\') &&
text.find_last_of(':') == 1;
+void SetFdToTextMode(int fd) {
+#ifdef _WIN32
if (_setmode(fd, _O_TEXT) == -1) {
// This should never happen, I think.
GOOGLE_LOG(WARNING) && &_setmode(& && fd && &, _O_TEXT): & && strerror(errno);
// (Text and binary are the same on non-Windows platforms.)
+void SetFdToBinaryMode(int fd) {
+#ifdef _WIN32
if (_setmode(fd, _O_BINARY) == -1) {
// This should never happen, I think.
GOOGLE_LOG(WARNING) && &_setmode(& && fd && &, _O_BINARY): & && strerror(errno);
// (Text and binary are the same on non-Windows platforms.)
+void AddTrailingSlash(string* path) {
if (!path-&empty() && path-&at(path-&size() - 1) != '/') {
path-&push_back('/');
+bool VerifyDirectoryExists(const string& path) {
if (path.empty())
if (access(path.c_str(), F_OK) == -1) {
cerr && path && &: & && strerror(errno) &&
+// Try to create the parent directory of the given file, creating the parent's
+// parent if necessary, and so on.
The full file name is actually
+// (prefix + filename), but we assume |prefix| already exists and only create
+// directories listed in |filename|.
+bool TryCreateParentDirectory(const string& prefix, const string& filename) {
// Recursively create parent directories to the output file.
vector&string& parts = Split(filename, &/&, true);
string path_so_far =
for (int i = 0; i & parts.size() - 1; i++) {
path_so_far += parts[i];
if (mkdir(path_so_far.c_str(), 0777) != 0) {
if (errno != EEXIST) {
cerr && filename && &: while trying to create directory &
&& path_so_far && &: & && strerror(errno) &&
path_so_far += '/';
// namespace
+// A MultiFileErrorCollector that prints errors to stderr.
+class CommandLineInterface::ErrorPrinter : public MultiFileErrorCollector,
public io::ErrorCollector {
ErrorPrinter(ErrorFormat format, DiskSourceTree *tree = NULL)
: format_(format), tree_(tree) {}
~ErrorPrinter() {}
// implements MultiFileErrorCollector ------------------------------
void AddError(const string& filename, int line, int column,
const string& message) {
// Print full path when running under MSVS
if (format_ == CommandLineInterface::ERROR_FORMAT_MSVS &&
tree_ != NULL &&
tree_-&VirtualFileToDiskFile(filename, &dfile)) {
// Users typically expect 1-based line/column numbers, so we add 1
// to each here.
if (line != -1) {
// Allow for both GCC- and Visual-Studio-compatible output.
switch (format_) {
case CommandLineInterface::ERROR_FORMAT_GCC:
cerr && &:& && (line + 1) && &:& && (column + 1);
case CommandLineInterface::ERROR_FORMAT_MSVS:
cerr && &(& && (line + 1) && &) : error in column=& && (column + 1);
cerr && &: & && message &&
// implements io::ErrorCollector -----------------------------------
void AddError(int line, int column, const string& message) {
AddError(&input&, line, column, message);
+ private:
const ErrorFormat format_;
DiskSourceTree *tree_;
+// -------------------------------------------------------------------
+// A GeneratorContext implementation that buffers files in memory, then dumps
+// them all to disk on demand.
+class CommandLineInterface::GeneratorContextImpl : public GeneratorContext {
GeneratorContextImpl(const vector&const FileDescriptor*&& parsed_files);
~GeneratorContextImpl();
// Write all files in the directory to disk at the given output location,
// which must end in a '/'.
bool WriteAllToDisk(const string& prefix);
// Write the contents of this directory to a ZIP-format archive with the
// given name.
bool WriteAllToZip(const string& filename);
// Add a boilerplate META-INF/MANIFEST.MF file as required by the Java JAR
// format, unless one has already been written.
void AddJarManifest();
// implements GeneratorContext --------------------------------------
io::ZeroCopyOutputStream* Open(const string& filename);
io::ZeroCopyOutputStream* OpenForAppend(const string& filename);
io::ZeroCopyOutputStream* OpenForInsert(
const string& filename, const string& insertion_point);
void ListParsedFiles(vector&const FileDescriptor*&* output) {
*output = parsed_files_;
+ private:
friend class MemoryOutputS
// map instead of hash_map so that files are written in order (good when
// writing zips).
map&string, string*& files_;
const vector&const FileDescriptor*&& parsed_files_;
bool had_error_;
+class CommandLineInterface::MemoryOutputStream
: public io::ZeroCopyOutputStream {
MemoryOutputStream(GeneratorContextImpl* directory, const string& filename,
bool append_mode);
MemoryOutputStream(GeneratorContextImpl* directory, const string& filename,
const string& insertion_point);
virtual ~MemoryOutputStream();
// implements ZeroCopyOutputStream ---------------------------------
virtual bool Next(void** data, int* size) { return inner_-&Next(data, size); }
virtual void BackUp(int count)
inner_-&BackUp(count);
virtual int64 ByteCount() const
{ return inner_-&ByteCount();
+ private:
// Where to insert the string when it's done.
GeneratorContextImpl* directory_;
string filename_;
string insertion_point_;
// The string we're building.
string data_;
// Whether we should append the output stream to the existing file.
bool append_mode_;
// StringOutputStream writing to data_.
scoped_ptr&io::StringOutputStream& inner_;
+// -------------------------------------------------------------------
+CommandLineInterface::GeneratorContextImpl::GeneratorContextImpl(
const vector&const FileDescriptor*&& parsed_files)
: parsed_files_(parsed_files),
had_error_(false) {
+CommandLineInterface::GeneratorContextImpl::~GeneratorContextImpl() {
STLDeleteValues(&files_);
+bool CommandLineInterface::GeneratorContextImpl::WriteAllToDisk(
const string& prefix) {
if (had_error_) {
if (!VerifyDirectoryExists(prefix)) {
for (map&string, string*&::const_iterator iter = files_.begin();
iter != files_.end(); ++iter) {
const string& relative_filename = iter-&
const char* data = iter-&second-&data();
int size = iter-&second-&size();
if (!TryCreateParentDirectory(prefix, relative_filename)) {
string filename = prefix + relative_
// Create the output file.
file_descriptor =
open(filename.c_str(), O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, 0666);
} while (file_descriptor & 0 && errno == EINTR);
if (file_descriptor & 0) {
int error =
cerr && filename && &: & && strerror(error);
// Write the file.
while (size & 0) {
int write_
write_result = write(file_descriptor, data, size);
} while (write_result & 0 && errno == EINTR);
if (write_result &= 0) {
// Write error.
// FIXME(kenton):
According to the man page, if write() returns zero,
write() simply did not write anything.
unclear under what circumstances this might happen, but presumably
errno won't be set in this case.
I am confused as to how such an
event should be handled.
For now I'm treating it as an error,
since retrying seems like it could lead to an infinite loop.
suspect this never actually happens anyway.
if (write_result & 0) {
int error =
cerr && filename && &: write: & && strerror(error);
cerr && filename && &: write() returned zero?& &&
data += write_
size -= write_
if (close(file_descriptor) != 0) {
int error =
cerr && filename && &: close: & && strerror(error);
+bool CommandLineInterface::GeneratorContextImpl::WriteAllToZip(
const string& filename) {
if (had_error_) {
// Create the output file.
file_descriptor =
open(filename.c_str(), O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, 0666);
} while (file_descriptor & 0 && errno == EINTR);
if (file_descriptor & 0) {
int error =
cerr && filename && &: & && strerror(error);
// Create the ZipWriter
io::FileOutputStream stream(file_descriptor);
ZipWriter zip_writer(&stream);
for (map&string, string*&::const_iterator iter = files_.begin();
iter != files_.end(); ++iter) {
zip_writer.Write(iter-&first, *iter-&second);
zip_writer.WriteDirectory();
if (stream.GetErrno() != 0) {
cerr && filename && &: & && strerror(stream.GetErrno()) &&
if (!stream.Close()) {
cerr && filename && &: & && strerror(stream.GetErrno()) &&
+void CommandLineInterface::GeneratorContextImpl::AddJarManifest() {
string** map_slot = &files_[&META-INF/MANIFEST.MF&];
if (*map_slot == NULL) {
*map_slot = new string(
&Manifest-Version: 1.0\n&
&Created-By: 1.6.0 (protoc)\n&
+io::ZeroCopyOutputStream* CommandLineInterface::GeneratorContextImpl::Open(
const string& filename) {
return new MemoryOutputStream(this, filename, false);
+io::ZeroCopyOutputStream*
+CommandLineInterface::GeneratorContextImpl::OpenForAppend(
const string& filename) {
return new MemoryOutputStream(this, filename, true);
+io::ZeroCopyOutputStream*
+CommandLineInterface::GeneratorContextImpl::OpenForInsert(
const string& filename, const string& insertion_point) {
return new MemoryOutputStream(this, filename, insertion_point);
+// -------------------------------------------------------------------
+CommandLineInterface::MemoryOutputStream::MemoryOutputStream(
GeneratorContextImpl* directory, const string& filename, bool append_mode)
: directory_(directory),
filename_(filename),
append_mode_(append_mode),
inner_(new io::StringOutputStream(&data_)) {
+CommandLineInterface::MemoryOutputStream::MemoryOutputStream(
GeneratorContextImpl* directory, const string& filename,
const string& insertion_point)
: directory_(directory),
filename_(filename),
insertion_point_(insertion_point),
inner_(new io::StringOutputStream(&data_)) {
+CommandLineInterface::MemoryOutputStream::~MemoryOutputStream() {
// Make sure all data has been written.
inner_.reset();
// Insert into the directory.
string** map_slot = &directory_-&files_[filename_];
if (insertion_point_.empty()) {
// This was just a regular Open().
if (*map_slot != NULL) {
if (append_mode_) {
(*map_slot)-&append(data_);
cerr && filename_ && &: Tried to write the same file twice.& &&
directory_-&had_error_ =
*map_slot =
(*map_slot)-&swap(data_);
// This was an OpenForInsert().
// If the data doens't end with a clean line break, add one.
if (!data_.empty() && data_[data_.size() - 1] != '\n') {
data_.push_back('\n');
// Find the file we are going to insert into.
if (*map_slot == NULL) {
cerr && filename_ && &: Tried to insert into file that doesn't exist.&
directory_-&had_error_ =
string* target = *map_
// Find the insertion point.
string magic_string = strings::Substitute(
&@@protoc_insertion_point($0)&, insertion_point_);
string::size_type pos = target-&find(magic_string);
if (pos == string::npos) {
cerr && filename_ && &: insertion point \&& && insertion_point_
&& &\& not found.& &&
directory_-&had_error_ =
// Seek backwards to the beginning of the line, which is where we will
// insert the data.
Note that this has the effect of pushing the insertion
// point down, so the data is inserted before it.
This is intentional
// because it means that multiple insertions at the same point will end
// up in the expected order in the final output.
pos = target-&find_last_of('\n', pos);
if (pos == string::npos) {
// Insertion point is on the first line.
// Advance to character after '\n'.
// Extract indent.
string indent_(*target, pos, target-&find_first_not_of(& \t&, pos) - pos);
if (indent_.empty()) {
// No indent.
This makes things easier.
target-&insert(pos, data_);
// Calculate how much space we need.
int indent_size = 0;
for (int i = 0; i & data_.size(); i++) {
if (data_[i] == '\n') indent_size += indent_.size();
// Make a hole for it.
target-&insert(pos, data_.size() + indent_size, '\0');
// Now copy in the data.
string::size_type data_pos = 0;
char* target_ptr = string_as_array(target) +
while (data_pos & data_.size()) {
// Copy indent.
memcpy(target_ptr, indent_.data(), indent_.size());
target_ptr += indent_.size();
// Copy line from data_.
// We already guaranteed that data_ ends with a newline (above), so this
// search can't fail.
string::size_type line_length =
data_.find_first_of('\n', data_pos) + 1 - data_
memcpy(target_ptr, data_.data() + data_pos, line_length);
target_ptr += line_
data_pos += line_
GOOGLE_CHECK_EQ(target_ptr,
string_as_array(target) + pos + data_.size() + indent_size);
+// ===================================================================
+CommandLineInterface::CommandLineInterface()
: mode_(MODE_COMPILE),
print_mode_(PRINT_NONE),
error_format_(ERROR_FORMAT_GCC),
imports_in_descriptor_set_(false),
source_info_in_descriptor_set_(false),
disallow_services_(false),
inputs_are_proto_path_relative_(false) {}
+CommandLineInterface::~CommandLineInterface() {}
+void CommandLineInterface::RegisterGenerator(const string& flag_name,
CodeGenerator* generator,
const string& help_text) {
GeneratorI
info.flag_name = flag_
info.generator =
info.help_text = help_
generators_by_flag_name_[flag_name] =
+void CommandLineInterface::RegisterGenerator(const string& flag_name,
const string& option_flag_name,
CodeGenerator* generator,
const string& help_text) {
GeneratorI
info.flag_name = flag_
info.option_flag_name = option_flag_
info.generator =
info.help_text = help_
generators_by_flag_name_[flag_name] =
generators_by_option_name_[option_flag_name] =
+void CommandLineInterface::AllowPlugins(const string& exe_name_prefix) {
plugin_prefix_ = exe_name_
+int CommandLineInterface::Run(int argc, const char* const argv[]) {
switch (ParseArguments(argc, argv)) {
case PARSE_ARGUMENT_DONE_AND_EXIT:
case PARSE_ARGUMENT_FAIL:
case PARSE_ARGUMENT_DONE_AND_CONTINUE:
// Set up the source tree.
DiskSourceTree source_
for (int i = 0; i & proto_path_.size(); i++) {
source_tree.MapPath(proto_path_[i].first, proto_path_[i].second);
// Map input files to virtual paths if necessary.
if (!inputs_are_proto_path_relative_) {
if (!MakeInputsBeProtoPathRelative(&source_tree)) {
// Allocate the Importer.
ErrorPrinter error_collector(error_format_, &source_tree);
Importer importer(&source_tree, &error_collector);
vector&const FileDescriptor*& parsed_
// Parse each file.
for (int i = 0; i & input_files_.size(); i++) {
// Import the file.
importer.AddUnusedImportTrackFile(input_files_[i]);
const FileDescriptor* parsed_file = importer.Import(input_files_[i]);
importer.ClearUnusedImportTrackFiles();
if (parsed_file == NULL) return 1;
parsed_files.push_back(parsed_file);
// Enforce --disallow_services.
if (disallow_services_ && parsed_file-&service_count() & 0) {
cerr && parsed_file-&name() && &: This file contains services, but &
&--disallow_services was used.& &&
// We construct a separate GeneratorContext for each output location.
// that two code generators may output to the same location, in which case
// they should share a single GeneratorContext so that OpenForInsert() works.
typedef hash_map&string, GeneratorContextImpl*& GeneratorContextM
GeneratorContextMap output_
// Generate output.
if (mode_ == MODE_COMPILE) {
for (int i = 0; i & output_directives_.size(); i++) {
string output_location = output_directives_[i].output_
if (!HasSuffixString(output_location, &.zip&) &&
!HasSuffixString(output_location, &.jar&)) {
AddTrailingSlash(&output_location);
GeneratorContextImpl** map_slot = &output_directories[output_location];
if (*map_slot == NULL) {
// First time we've seen this output location.
*map_slot = new GeneratorContextImpl(parsed_files);
if (!GenerateOutput(parsed_files, output_directives_[i], *map_slot)) {
STLDeleteValues(&output_directories);
// Write all output to disk.
for (GeneratorContextMap::iterator iter = output_directories.begin();
iter != output_directories.end(); ++iter) {
const string& location = iter-&
GeneratorContextImpl* directory = iter-&
if (HasSuffixString(location, &/&)) {
if (!directory-&WriteAllToDisk(location)) {
STLDeleteValues(&output_directories);
if (HasSuffixString(location, &.jar&)) {
directory-&AddJarManifest();
if (!directory-&WriteAllToZip(location)) {
STLDeleteValues(&output_directories);
STLDeleteValues(&output_directories);
if (!descriptor_set_name_.empty()) {
if (!WriteDescriptorSet(parsed_files)) {
if (mode_ == MODE_ENCODE || mode_ == MODE_DECODE) {
if (codec_type_.empty()) {
Define an EmptyMessage type to use for decoding.
DescriptorP
FileDescriptorP
file.set_name(&empty_message.proto&);
file.add_message_type()-&set_name(&EmptyMessage&);
GOOGLE_CHECK(pool.BuildFile(file) != NULL);
codec_type_ = &EmptyMessage&;
if (!EncodeOrDecode(&pool)) {
if (!EncodeOrDecode(importer.pool())) {
if (mode_ == MODE_PRINT) {
switch (print_mode_) {
case PRINT_FREE_FIELDS:
for (int i = 0; i & parsed_files.size(); ++i) {
const FileDescriptor* fd = parsed_files[i];
for (int j = 0; j & fd-&message_type_count(); ++j) {
PrintFreeFieldNumbers(fd-&message_type(j));
case PRINT_NONE:
GOOGLE_LOG(ERROR) && &If the code reaches here, it usually means a bug of &
&flag parsing in the CommonadLineInterface.&;
// Do not add a default case.
+void CommandLineInterface::Clear() {
// Clear all members that are set by Run().
Note that we must not clear
// members which are set by other methods before Run() is called.
executable_name_.clear();
proto_path_.clear();
input_files_.clear();
output_directives_.clear();
codec_type_.clear();
descriptor_set_name_.clear();
mode_ = MODE_COMPILE;
print_mode_ = PRINT_NONE;
imports_in_descriptor_set_ =
source_info_in_descriptor_set_ =
disallow_services_ =
+bool CommandLineInterface::MakeInputsBeProtoPathRelative(
DiskSourceTree* source_tree) {
for (int i = 0; i & input_files_.size(); i++) {
string virtual_file, shadowing_disk_
switch (source_tree-&DiskFileToVirtualFile(
input_files_[i], &virtual_file, &shadowing_disk_file)) {
case DiskSourceTree::SUCCESS:
input_files_[i] = virtual_
case DiskSourceTree::SHADOWED:
cerr && input_files_[i] && &: Input is shadowed in the --proto_path &
&by \&& && shadowing_disk_file && &\&.
Either use the latter &
&file as your input or reorder the --proto_path so that the &
&former file's location comes first.& &&
case DiskSourceTree::CANNOT_OPEN:
cerr && input_files_[i] && &: & && strerror(errno) &&
case DiskSourceTree::NO_MAPPING:
// First check if the file exists at all.
if (access(input_files_[i].c_str(), F_OK) & 0) {
// File does not even exist.
cerr && input_files_[i] && &: & && strerror(ENOENT) &&
cerr && input_files_[i] && &: File does not reside within any path &
&specified using --proto_path (or -I).
You must specify a &
&--proto_path which encompasses this file.
Note that the &
&proto_path must be an exact prefix of the .proto file &
&names -- protoc is too dumb to figure out when two paths &
&(e.g. absolute and relative) are equivalent (it's harder &
&than you think).& &&
+CommandLineInterface::ParseArgumentStatus
+CommandLineInterface::ParseArguments(int argc, const char* const argv[]) {
executable_name_ = argv[0];
// Iterate through all arguments and parse them.
for (int i = 1; i & i++) {
string name,
if (ParseArgument(argv[i], &name, &value)) {
// Returned true =& Use the next argument as the flag value.
if (i + 1 == argc || argv[i+1][0] == '-') {
cerr && &Missing value for flag: & && name &&
if (name == &--decode&) {
cerr && &To decode an unknown message, use --decode_raw.& &&
return PARSE_ARGUMENT_FAIL;
value = argv[i];
ParseArgumentStatus status = InterpretArgument(name, value);
if (status != PARSE_ARGUMENT_DONE_AND_CONTINUE)
// If no --proto_path was given, use the current working directory.
if (proto_path_.empty()) {
// Don't use make_pair as the old/default standard library on Solaris
// doesn't support it without explicit template parameters, which are
// incompatible with C++0x's make_pair.
proto_path_.push_back(pair&string, string&(&&, &.&));
// Check some errror cases.
bool decoding_raw = (mode_ == MODE_DECODE) && codec_type_.empty();
if (decoding_raw && !input_files_.empty()) {
cerr && &When using --decode_raw, no input files should be given.& &&
return PARSE_ARGUMENT_FAIL;
} else if (!decoding_raw && input_files_.empty()) {
cerr && &Missing input file.& &&
return PARSE_ARGUMENT_FAIL;
if (mode_ == MODE_COMPILE && output_directives_.empty() &&
descriptor_set_name_.empty()) {
cerr && &Missing output directives.& &&
return PARSE_ARGUMENT_FAIL;
if (imports_in_descriptor_set_ && descriptor_set_name_.empty()) {
cerr && &--include_imports only makes sense when combined with &
&--descriptor_set_out.& &&
if (source_info_in_descriptor_set_ && descriptor_set_name_.empty()) {
cerr && &--include_source_info only makes sense when combined with &
&--descriptor_set_out.& &&
return PARSE_ARGUMENT_DONE_AND_CONTINUE;
+bool CommandLineInterface::ParseArgument(const char* arg,
string* name, string* value) {
bool parsed_value =
if (arg[0] != '-') {
// Not a flag.
name-&clear();
parsed_value =
} else if (arg[1] == '-') {
// Two dashes:
Multi-character name, with '=' separating name and
const char* equals_pos = strchr(arg, '=');
if (equals_pos != NULL) {
*name = string(arg, equals_pos - arg);
*value = equals_pos + 1;
parsed_value =
// One dash:
One-character name, all subsequent characters are the
if (arg[1] == '\0') {
// arg is just &-&.
We treat this as an input file, except that at
// present this will just lead to a &file not found& error.
name-&clear();
parsed_value =
*name = string(arg, 2);
*value = arg + 2;
parsed_value = !value-&empty();
// Need to return true iff the next arg should be used as the value for this
// one, false otherwise.
if (parsed_value) {
// We already parsed a value for this flag.
if (*name == &-h& || *name == &--help& ||
*name == &--disallow_services& ||
*name == &--include_imports& ||
*name == &--include_source_info& ||
*name == &--version& ||
*name == &--decode_raw& ||
*name == &--print_free_field_numbers&) {
These are the only flags that don't take a value.
They probably should not be hard-coded like this but for now it's
not worth doing better.
// Next argument is the flag value.
+CommandLineInterface::ParseArgumentStatus
+CommandLineInterface::InterpretArgument(const string& name,
const string& value) {
if (name.empty()) {
// Not a flag.
Just a filename.
if (value.empty()) {
cerr && &You seem to have passed an empty string as one of the &
&arguments to & && executable_name_ && &.
This is actually &
&sort of hard to do.
Unfortunately it is not valid &
&input so the program is going to die now.& &&
return PARSE_ARGUMENT_FAIL;
input_files_.push_back(value);
} else if (name == &-I& || name == &--proto_path&) {
// Java's -classpath (and some other languages) delimits path components
// with colons.
Let's accept that syntax too just to make things more
// intuitive.
vector&string& parts = Split(
value, kPathSeparator, true);
for (int i = 0; i & parts.size(); i++) {
string virtual_
string disk_
string::size_type equals_pos = parts[i].find_first_of('=');
if (equals_pos == string::npos) {
virtual_path = &&;
disk_path = parts[i];
virtual_path = parts[i].substr(0, equals_pos);
disk_path = parts[i].substr(equals_pos + 1);
if (disk_path.empty()) {
cerr && &--proto_path passed empty directory name.
(Use \&.\& for &
¤t directory.)& &&
return PARSE_ARGUMENT_FAIL;
// Make sure disk path exists, warn otherwise.
if (access(disk_path.c_str(), F_OK) & 0) {
cerr && disk_path && &: warning: directory does not exist.& &&
// Don't use make_pair as the old/default standard library on Solaris
// doesn't support it without explicit template parameters, which are
// incompatible with C++0x's make_pair.
proto_path_.push_back(pair&string, string&(virtual_path, disk_path));
} else if (name == &-o& || name == &--descriptor_set_out&) {
if (!descriptor_set_name_.empty()) {
cerr && name && & may only be passed once.& &&
return PARSE_ARGUMENT_FAIL;
if (value.empty()) {
cerr && name && & requires a non-empty value.& &&
return PARSE_ARGUMENT_FAIL;
if (mode_ != MODE_COMPILE) {
cerr && &Cannot use --encode or --decode and generate descriptors at the &
&same time.& &&
return PARSE_ARGUMENT_FAIL;
descriptor_set_name_ =
} else if (name == &--include_imports&) {
if (imports_in_descriptor_set_) {
cerr && name && & may only be passed once.& &&
return PARSE_ARGUMENT_FAIL;
imports_in_descriptor_set_ =
} else if (name == &--include_source_info&) {
if (source_info_in_descriptor_set_) {
cerr && name && & may only be passed once.& &&
return PARSE_ARGUMENT_FAIL;
source_info_in_descriptor_set_ =
} else if (name == &-h& || name == &--help&) {
PrintHelpText();
return PARSE_ARGUMENT_DONE_AND_EXIT;
// Exit without running compiler.
} else if (name == &--version&) {
if (!version_info_.empty()) {
cout && version_info_ &&
cout && &libprotoc &
&& protobuf::internal::VersionString(GOOGLE_PROTOBUF_VERSION)
return PARSE_ARGUMENT_DONE_AND_EXIT;
// Exit without running compiler.
} else if (name == &--disallow_services&) {
disallow_services_ =
} else if (name == &--encode& || name == &--decode& ||
name == &--decode_raw&) {
if (mode_ != MODE_COMPILE) {
cerr && &Only one of --encode and --decode can be specified.& &&
return PARSE_ARGUMENT_FAIL;
if (!output_directives_.empty() || !descriptor_set_name_.empty()) {
cerr && &Cannot use & && name
&& & and generate code or descriptors at the same time.& &&
return PARSE_ARGUMENT_FAIL;
mode_ = (name == &--encode&) ? MODE_ENCODE : MODE_DECODE;
if (value.empty() && name != &--decode_raw&) {
cerr && &Type name for & && name && & cannot be blank.& &&
if (name == &--decode&) {
cerr && &To decode an unknown message, use --decode_raw.& &&
return PARSE_ARGUMENT_FAIL;
} else if (!value.empty() && name == &--decode_raw&) {
cerr && &--decode_raw does not take a parameter.& &&
return PARSE_ARGUMENT_FAIL;
codec_type_ =
} else if (name == &--error_format&) {
if (value == &gcc&) {
error_format_ = ERROR_FORMAT_GCC;
} else if (value == &msvs&) {
error_format_ = ERROR_FORMAT_MSVS;
cerr && &Unknown error format: & && value &&
return PARSE_ARGUMENT_FAIL;
} else if (name == &--plugin&) {
if (plugin_prefix_.empty()) {
cerr && &This compiler does not support plugins.& &&
return PARSE_ARGUMENT_FAIL;
string plugin_
string::size_type equals_pos = value.find_first_of('=');
if (equals_pos == string::npos) {
// Use the basename of the file.
string::size_type slash_pos = value.find_last_of('/');
if (slash_pos == string::npos) {
plugin_name =
plugin_name = value.substr(slash_pos + 1);
plugin_name = value.substr(0, equals_pos);
path = value.substr(equals_pos + 1);
plugins_[plugin_name] =
} else if (name == &--print_free_field_numbers&) {
if (mode_ != MODE_COMPILE) {
cerr && &Cannot use & && name && & and use --encode, --decode or print &
&& &other info at the same time.& &&
return PARSE_ARGUMENT_FAIL;
if (!output_directives_.empty() || !descriptor_set_name_.empty()) {
cerr && &Cannot use & && name
&& & and generate code or descriptors at the same time.& &&
return PARSE_ARGUMENT_FAIL;
mode_ = MODE_PRINT;
print_mode_ = PRINT_FREE_FIELDS;
// Some other flag.
Look it up in the generators list.
const GeneratorInfo* generator_info =
FindOrNull(generators_by_flag_name_, name);
if (generator_info == NULL &&
(plugin_prefix_.empty() || !HasSuffixString(name, &_out&))) {
// Check if it's a generator option flag.
generator_info = FindOrNull(generators_by_option_name_, name);
if (generator_info == NULL) {
cerr && &Unknown flag: & && name &&
return PARSE_ARGUMENT_FAIL;
string* parameters = &generator_parameters_[generator_info-&flag_name];
if (!parameters-&empty()) {
parameters-&append(&,&);
parameters-&append(value);
// It's an output flag.
Add it to the output directives.
if (mode_ != MODE_COMPILE) {
cerr && &Cannot use --encode, --decode or print .proto info and &
&generate code at the same time.& &&
return PARSE_ARGUMENT_FAIL;
directive.name =
if (generator_info == NULL) {
directive.generator = NULL;
directive.generator = generator_info-&
// Split value at ':' to separate the generator parameter from the
// filename.
However, avoid doing this if the colon is part of a valid
// Windows-style absolute path.
string::size_type colon_pos = value.find_first_of(':');
if (colon_pos == string::npos || IsWindowsAbsolutePath(value)) {
directive.output_location =
directive.parameter = value.substr(0, colon_pos);
directive.output_location = value.substr(colon_pos + 1);
output_directives_.push_back(directive);
return PARSE_ARGUMENT_DONE_AND_CONTINUE;
+void CommandLineInterface::PrintHelpText() {
line wrapping would be uglier.
+&Usage: & && executable_name_ && & [OPTION] PROTO_FILES\n&
+&Parse PROTO_FILES and generate output based on the options given:\n&
-IPATH, --proto_path=PATH
Specify the directory in which to search for\n&
May be spec\n&
directories will be searched in order.
given, the current working directory is used.\n&
Show version info and exit.\n&
-h, --help
Show this text and exit.\n&
--encode=MESSAGE_TYPE
Read a text-format message of the given type\n&
from standard input and write it in binary\n&
to standard output.
The message type must\n&
be defined in PROTO_FILES or their imports.\n&
--decode=MESSAGE_TYPE
Read a binary message of the given type from\n&
standard input and write it in text format\n&
to standard output.
The message type must\n&
be defined in PROTO_FILES or their imports.\n&
--decode_raw
Read an arbitrary protocol message from\n&
standard input and write the raw tag/value\n&
pairs in text format to standard output.
PROTO_FILES should be given when using this\n&
Writes a FileDescriptorSet (a protocol buffer,\n&
--descriptor_set_out=FILE defined in descriptor.proto) containing all of\n&
the input files to FILE.\n&
--include_imports
When using --descriptor_set_out, also include\n&
all dependencies of the input files in the\n&
set, so that the set is self-contained.\n&
--include_source_info
When using --descriptor_set_out, do not strip\n&
SourceCodeInfo from the FileDescriptorProto.\n&
This results in vastly larger descriptors that\n&
include information about the original\n&
location of each decl in the source file as\n&
well as surrounding comments.\n&
--error_format=FORMAT
Set the format in which to print errors.\n&
FORMAT may be 'gcc' (the default) or 'msvs'\n&
(Microsoft Visual Studio format).\n&
--print_free_field_numbers
Print the free field numbers of the messages\n&
defined in the given proto files. Groups share\n&
the same field number space with the parent \n&
message. Extension ranges are counted as \n&
occupied fields numbers.&
if (!plugin_prefix_.empty()) {
--plugin=EXECUTABLE
Specifies a plugin executable to use.\n&
Normally, protoc searches the PATH for\n&
plugins, but you may specify additional\n&
executables not in the path using this flag.\n&
Additionally, EXECUTABLE may be of the form\n&
NAME=PATH, in which case the given plugin name\n&
is mapped to the given executable even if\n&
the executable's own name differs.& &&
for (GeneratorMap::iterator iter = generators_by_flag_name_.begin();
iter != generators_by_flag_name_.end(); ++iter) {
// FIXME(kenton):
If the text is long enough it will wrap, which is ugly,
but fixing this nicely (e.g. splitting on spaces) is probably more
trouble than it's worth.
& && iter-&first && &=OUT_DIR &
&& string(19 - iter-&first.size(), ' ')
// Spaces for alignment.
&& iter-&second.help_text &&
+bool CommandLineInterface::GenerateOutput(
const vector&const FileDescriptor*&& parsed_files,
const OutputDirective& output_directive,
GeneratorContext* generator_context) {
// Call the generator.
if (output_directive.generator == NULL) {
// This is a plugin.
GOOGLE_CHECK(HasPrefixString(output_directive.name, &--&) &&
HasSuffixString(output_directive.name, &_out&))
&& &Bad name for plugin generator: & && output_directive.
// Strip the &--& and &_out& and add the plugin prefix.
string plugin_name = plugin_prefix_ + &gen-& +
output_directive.name.substr(2, output_directive.name.size() - 6);
if (!GeneratePluginOutput(parsed_files, plugin_name,
output_directive.parameter,
generator_context, &error)) {
cerr && output_directive.name && &: & && error &&
// Regular generator.
string parameters = output_directive.
if (!generator_parameters_[output_directive.name].empty()) {
if (!parameters.empty()) {
parameters.append(&,&);
parameters.append(generator_parameters_[output_directive.name]);
for (int i = 0; i & parsed_files.size(); i++) {
if (!output_directive.generator-&Generate(parsed_files[i], parameters,
generator_context, &error)) {
// Generator returned an error.
cerr && output_directive.name && &: & && parsed_files[i]-&name() && &: &
&& error &&
+bool CommandLineInterface::GeneratePluginOutput(
const vector&const FileDescriptor*&& parsed_files,
const string& plugin_name,
const string& parameter,
GeneratorContext* generator_context,
string* error) {
CodeGeneratorR
CodeGeneratorR
// Build the request.
if (!parameter.empty()) {
request.set_parameter(parameter);
set&const FileDescriptor*& already_
for (int i = 0; i & parsed_files.size(); i++) {
request.add_file_to_generate(parsed_files[i]-&name());
GetTransitiveDependencies(parsed_files[i],
// Include source code info.
&already_seen, request.mutable_proto_file());
// Invoke the plugin.
if (plugins_.count(plugin_name) & 0) {
subprocess.Start(plugins_[plugin_name], Subprocess::EXACT_NAME);
subprocess.Start(plugin_name, Subprocess::SEARCH_PATH);
string communicate_
if (!municate(request, &response, &communicate_error)) {
*error = strings::Substitute(&$0: $1&, plugin_name, communicate_error);
// Write the files.
We do this even if there was a generator error in order
// to match the behavior of a compiled-in generator.
scoped_ptr&io::ZeroCopyOutputStream& current_
for (int i = 0; i & response.file_size(); i++) {
const CodeGeneratorResponse::File& output_file = response.file(i);
if (!output_file.insertion_point().empty()) {
// Open a file for insert.
// We reset current_output to NULL first so that the old file is closed
// before the new one is opened.
current_output.reset();
current_output.reset(generator_context-&OpenForInsert(
output_file.name(), output_file.insertion_point()));
} else if (!output_file.name().empty()) {
// Starting a new file.
// We reset current_output to NULL first so that the old file is closed
// before the new one is opened.
current_output.reset();
current_output.reset(generator_context-&Open(output_file.name()));
} else if (current_output == NULL) {
*error = strings::Substitute(
&$0: First file chunk returned by plugin did not specify a file name.&,
plugin_name);
// Use CodedOutputSt otherwise we'd need to provide
// our own buffer-copying loop.
io::CodedOutputStream writer(current_output.get());
writer.WriteString(output_file.content());
// Check for errors.
if (!response.error().empty()) {
// Generator returned an error.
*error = response.error();
+bool CommandLineInterface::EncodeOrDecode(const DescriptorPool* pool) {
// Look up the type.
const Descriptor* type = pool-&FindMessageTypeByName(codec_type_);
if (type == NULL) {
cerr && &Type not defined: & && codec_type_ &&
DynamicMessageFactory dynamic_factory(pool);
scoped_ptr&Message& message(dynamic_factory.GetPrototype(type)-&New());
if (mode_ == MODE_ENCODE) {
SetFdToTextMode(STDIN_FILENO);
SetFdToBinaryMode(STDOUT_FILENO);
SetFdToBinaryMode(STDIN_FILENO);
SetFdToTextMode(STDOUT_FILENO);
io::FileInputStream in(STDIN_FILENO);
io::FileOutputStream out(STDOUT_FILENO);
if (mode_ == MODE_ENCODE) {
// Input is text.
ErrorPrinter error_collector(error_format_);
TextFormat::P
parser.RecordErrorsTo(&error_collector);
parser.AllowPartialMessage(true);
if (!parser.Parse(&in, message.get())) {
cerr && &Failed to parse input.& &&
// Input is binary.
if (!message-&ParsePartialFromZeroCopyStream(&in)) {
cerr && &Failed to parse input.& &&
if (!message-&IsInitialized()) {
cerr && &warning:
Input message is missing required fields:
&& message-&InitializationErrorString() &&
if (mode_ == MODE_ENCODE) {
// Output is binary.
if (!message-&SerializePartialToZeroCopyStream(&out)) {
cerr && &output: I/O error.& &&
// Output is text.
if (!TextFormat::Print(*message, &out)) {
cerr && &output: I/O error.& &&
+bool CommandLineInterface::WriteDescriptorSet(
const vector&const FileDescriptor*& parsed_files) {
FileDescriptorSet file_
if (imports_in_descriptor_set_) {
set&const FileDescriptor*& already_
for (int i = 0; i & parsed_files.size(); i++) {
GetTransitiveDependencies(parsed_files[i],
source_info_in_descriptor_set_,
&already_seen, file_set.mutable_file());
for (int i = 0; i & parsed_files.size(); i++) {
FileDescriptorProto* file_proto = file_set.add_file();
parsed_files[i]-&CopyTo(file_proto);
if (source_info_in_descriptor_set_) {
parsed_files[i]-&CopySourceCodeInfoTo(file_proto);
fd = open(descriptor_set_name_.c_str(),
O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, 0666);
} while (fd & 0 && errno == EINTR);
if (fd & 0) {
perror(descriptor_set_name_.c_str());
io::FileOutputStream out(fd);
if (!file_set.SerializeToZeroCopyStream(&out)) {
cerr && descriptor_set_name_ && &: & && strerror(out.GetErrno()) &&
out.Close();
if (!out.Close()) {
cerr && descriptor_set_name_ && &: & && strerror(out.GetErrno()) &&
+void CommandLineInterface::GetTransitiveDependencies(
const FileDescriptor* file, bool include_source_code_info,
set&const FileDescriptor*&* already_seen,
RepeatedPtrField&FileDescriptorProto&* output) {
if (!already_seen-&insert(file).second) {
// Already saw this file.
// Add all dependencies.
for (int i = 0; i & file-&dependency_count(); i++) {
GetTransitiveDependencies(file-&dependency(i), include_source_code_info,
already_seen, output);
// Add this file.
FileDescriptorProto* new_descriptor = output-&Add();
file-&CopyTo(new_descriptor);
if (include_source_code_info) {
file-&CopySourceCodeInfoTo(new_descriptor);
+namespace {
+// Utility function for PrintFreeFieldNumbers.
+// Stores occupied ranges into the ranges parameter, and next level of sub
+// message types into the nested_messages parameter.
The FieldRange is left
+// inclusive, right exclusive. i.e. [a, b).
+// Nested Messages:
+// Note that it only stores the nested message type, iff the nested type is
+// either a direct child of the given descriptor, or the nested type is a
+// decendent of the given descriptor and all the nodes between the
+// nested type and the given descriptor are group types. e.g.
+// message Foo {
message Bar {
message NestedBar {}
group Baz = 1 {
group NestedBazGroup = 2 {
message Quz {
message NestedQuz {}
message NestedBaz {}
+// In this case, Bar, Quz and NestedBaz will be added into the nested types.
+// Since free field numbers of group types will not be printed, this makes sure
+// the nested message types in groups will not be dropped. The nested_messages
+// parameter will contain the direct children (when groups are ignored in the
+// tree) of the given descriptor for the caller to traverse. The declaration
+// order of the nested messages is also preserved.
+typedef pair&int, int& FieldR
+void GatherOccupiedFieldRanges(const Descriptor* descriptor,
set&FieldRange&* ranges,
vector&const Descriptor*&* nested_messages) {
set&const Descriptor*&
for (int i = 0; i & descriptor-&field_count(); ++i) {
const FieldDescriptor* fd = descriptor-&field(i);
ranges-&insert(FieldRange(fd-&number(), fd-&number() + 1));
if (fd-&type() == FieldDescriptor::TYPE_GROUP) {
groups.insert(fd-&message_type());
for (int i = 0; i & descriptor-&extension_range_count(); ++i) {
ranges-&insert(FieldRange(descriptor-&extension_range(i)-&start,
descriptor-&extension_range(i)-&end));
// Handle the nested messages/groups in declaration order to make it
// post-order strict.
for (int i = 0; i & descriptor-&nested_type_count(); ++i) {
const Descriptor* nested_desc = descriptor-&nested_type(i);
if (groups.find(nested_desc) != groups.end()) {
GatherOccupiedFieldRanges(nested_desc, ranges, nested_messages);
nested_messages-&push_back(nested_desc);
+// Utility function for PrintFreeFieldNumbers.
+// Actually prints the formatted free field numbers for given message name and
+// occupied ranges.
+void FormatFreeFieldNumbers(const string& name,
const set&FieldRange&& ranges) {
StringAppendF(&output, &%-35s free:&, name.c_str());
int next_free_number = 1;
for (set&FieldRange&::iterator i = ranges.begin();
i != ranges.end(); ++i) {
// This happens when groups re-use parent field numbers, in which
// case we skip the FieldRange entirely.
if (next_free_number &= i-&second)
if (next_free_number & i-&first) {
if (next_free_number + 1 == i-&first) {
// Singleton
StringAppendF(&output, & %d&, next_free_number);
StringAppendF(&output, & %d-%d&, next_free_number, i-&first - 1);
next_free_number = i-&
if (next_free_number &= FieldDescriptor::kMaxNumber) {
StringAppendF(&output, & %d-INF&, next_free_number);
cout && output &&
// namespace
+void CommandLineInterface::PrintFreeFieldNumbers(
const Descriptor* descriptor) {
set&FieldRange&
vector&const Descriptor*& nested_
GatherOccupiedFieldRanges(descriptor, &ranges, &nested_messages);
for (int i = 0; i & nested_messages.size(); ++i) {
PrintFreeFieldNumbers(nested_messages[i]);
FormatFreeFieldNumbers(descriptor-&full_name(), ranges);
// namespace compiler
// namespace protobuf
// namespace google
(C)&&2013&&Alibaba&&Inc.&&All&&rights&&resvered.
Powered by
