public void testHBaseSerDeCustomStructValue() throws IOException, SerDeException { byte[] cfa = "cola".getBytes(); byte[] qualStruct = "struct".getBytes(); TestStruct testStruct = new TestStruct("A", "B", "C", false, (byte) 0); byte[] key = testStruct.getBytes(); // Data List<Cell> kvs = new ArrayList<Cell>(); byte[] testData = testStruct.getBytes(); kvs.add(new KeyValue(key, cfa, qualStruct, testData)); Result r = Result.create(kvs); byte[] putKey = testStruct.getBytesWithDelimiters(); Put p = new Put(putKey); // Post serialization, separators are automatically inserted between different fields in the // struct. Currently there is not way to disable that. So the work around here is to pad the // data with the separator bytes before creating a "Put" object p.add(new KeyValue(putKey, cfa, qualStruct, Bytes.padTail(testData, 2))); // Create, initialize, and test the SerDe HBaseSerDe serDe = new HBaseSerDe(); Configuration conf = new Configuration(); Properties tbl = createPropertiesForValueStruct(); serDe.initialize(conf, tbl); deserializeAndSerializeHBaseValueStruct(serDe, r, p); }
@Override public int toBytes(Object object, byte[] bytes, int offset) { if (object == null) { // Create the byte[] of size MAX_TIMESTAMP_BYTES if(bytes.length != getByteSize()) { bytes = Bytes.padTail(bytes, (getByteSize() - bytes.length)); } PDate.INSTANCE.getCodec().encodeLong(0l, bytes, offset); Bytes.putInt(bytes, offset + Bytes.SIZEOF_LONG, 0); return getByteSize(); } java.sql.Timestamp value = (java.sql.Timestamp) object; // For Timestamp, the getTime() method includes milliseconds that may // be stored in the nanos part as well. DateUtil.getCodecFor(this).encodeLong(value.getTime(), bytes, offset); /* * By not getting the stuff that got spilled over from the millis part, * it leaves the timestamp's byte representation saner - 8 bytes of millis | 4 bytes of nanos. * Also, it enables timestamp bytes to be directly compared with date/time bytes. */ Bytes.putInt(bytes, offset + Bytes.SIZEOF_LONG, value.getNanos() % MAX_NANOS_VALUE_EXCLUSIVE); return getByteSize(); }
@Override public float getProgress() { if (currRow_ == null || currRow_.length == 0 || endRow_.length == 0 || endRow_ == HConstants.LAST_ROW) { return 0; } byte[] lastPadded = currRow_; if (currRow_.length < endRow_.length) { lastPadded = Bytes.padTail(currRow_, endRow_.length - currRow_.length); } if (currRow_.length < startRow_.length) { lastPadded = Bytes.padTail(currRow_, startRow_.length - currRow_.length); } byte [] prependHeader = {1, 0}; BigInteger bigLastRow = new BigInteger(Bytes.add(prependHeader, lastPadded)); if (bigLastRow.compareTo(bigEnd_) > 0) { return progressSoFar_; } BigDecimal processed = new BigDecimal(bigLastRow.subtract(bigStart_)); try { BigDecimal progress = processed.setScale(3).divide(bigRange_, BigDecimal.ROUND_HALF_DOWN); progressSoFar_ = progress.floatValue(); return progressSoFar_; } catch (java.lang.ArithmeticException e) { return 0; } }
@Override public void setScan(Scan scan) { super.setScan(scan); startRow_ = scan.getStartRow(); endRow_ = scan.getStopRow(); byte[] startPadded; byte[] endPadded; if (startRow_.length < endRow_.length) { startPadded = Bytes.padTail(startRow_, endRow_.length - startRow_.length); endPadded = endRow_; } else if (endRow_.length < startRow_.length) { startPadded = startRow_; endPadded = Bytes.padTail(endRow_, startRow_.length - endRow_.length); } else { startPadded = startRow_; endPadded = endRow_; } currRow_ = startRow_; byte [] prependHeader = {1, 0}; bigStart_ = new BigInteger(Bytes.add(prependHeader, startPadded)); bigEnd_ = new BigInteger(Bytes.add(prependHeader, endPadded)); bigRange_ = new BigDecimal(bigEnd_.subtract(bigStart_)); LOG.info("setScan with ranges: " + bigStart_ + " - " + bigEnd_ + " ( " + bigRange_ + ")"); }
@Override public int toBytes(Object object, byte[] bytes, int offset) { if (object == null) { // Create the byte[] of size MAX_TIMESTAMP_BYTES if(bytes.length != getByteSize()) { bytes = Bytes.padTail(bytes, (getByteSize() - bytes.length)); } PDate.INSTANCE.getCodec().encodeLong(0l, bytes, offset); Bytes.putInt(bytes, offset + Bytes.SIZEOF_LONG, 0); return getByteSize(); } java.sql.Timestamp value = (java.sql.Timestamp) object; // For Timestamp, the getTime() method includes milliseconds that may // be stored in the nanos part as well. DateUtil.getCodecFor(this).encodeLong(value.getTime(), bytes, offset); /* * By not getting the stuff that got spilled over from the millis part, * it leaves the timestamp's byte representation saner - 8 bytes of millis | 4 bytes of nanos. * Also, it enables timestamp bytes to be directly compared with date/time bytes. */ Bytes.putInt(bytes, offset + Bytes.SIZEOF_LONG, value.getNanos() % MAX_NANOS_VALUE_EXCLUSIVE); return getByteSize(); }
@Override public int toBytes(Object object, byte[] bytes, int offset) { if (object == null) { // Create the byte[] of size MAX_TIMESTAMP_BYTES if(bytes.length != getByteSize()) { bytes = Bytes.padTail(bytes, (getByteSize() - bytes.length)); } PDate.INSTANCE.getCodec().encodeLong(0l, bytes, offset); Bytes.putInt(bytes, offset + Bytes.SIZEOF_LONG, 0); return getByteSize(); } java.sql.Timestamp value = (java.sql.Timestamp) object; // For Timestamp, the getTime() method includes milliseconds that may // be stored in the nanos part as well. DateUtil.getCodecFor(this).encodeLong(value.getTime(), bytes, offset); /* * By not getting the stuff that got spilled over from the millis part, * it leaves the timestamp's byte representation saner - 8 bytes of millis | 4 bytes of nanos. * Also, it enables timestamp bytes to be directly compared with date/time bytes. */ Bytes.putInt(bytes, offset + Bytes.SIZEOF_LONG, value.getNanos() % MAX_NANOS_VALUE_EXCLUSIVE); return getByteSize(); }