private void checkTypeConsistency(IVariableTypeEnvironment expected, List<LogicalVariable> expectedVariables, IVariableTypeEnvironment actual, List<LogicalVariable> actualVariables) throws AlgebricksException { for (int i = 0; i < expectedVariables.size(); i++) { Object expectedType = expected.getVarType(expectedVariables.get(i)); Object actualType = actual.getVarType(actualVariables.get(i)); if (!expectedType.equals(actualType)) { if (AlgebricksConfig.ALGEBRICKS_LOGGER.isWarnEnabled()) { AlgebricksConfig.ALGEBRICKS_LOGGER .warn("Type of two variables are not equal." + expectedVariables.get(i) + " is of type: " + expectedType + actualVariables.get(i) + " is of type: " + actualType); } } } }
private Object getType(IVariableTypeEnvironment env, LogicalVariable var) throws AlgebricksException { Object type = env.getVarType(var); if (type == null) { throw new AlgebricksException("Failed typing union operator: no type for variable " + var); } return type; } }
public static IBinaryHashFunctionFamily[] variablesToBinaryHashFunctionFamilies( Collection<LogicalVariable> varLogical, IVariableTypeEnvironment env, JobGenContext context) throws AlgebricksException { IBinaryHashFunctionFamily[] funFamilies = new IBinaryHashFunctionFamily[varLogical.size()]; int i = 0; IBinaryHashFunctionFamilyProvider bhffProvider = context.getBinaryHashFunctionFamilyProvider(); for (LogicalVariable var : varLogical) { Object type = env.getVarType(var); funFamilies[i++] = bhffProvider.getBinaryHashFunctionFamily(type); } return funFamilies; }
public static IBinaryComparatorFactory[] variablesToAscBinaryComparatorFactories(List<LogicalVariable> varLogical, int start, int size, IVariableTypeEnvironment env, JobGenContext context) throws AlgebricksException { IBinaryComparatorFactory[] compFactories = new IBinaryComparatorFactory[size]; IBinaryComparatorFactoryProvider bcfProvider = context.getBinaryComparatorFactoryProvider(); for (int i = 0; i < size; i++) { Object type = env.getVarType(varLogical.get(start + i)); compFactories[i] = bcfProvider.getBinaryComparatorFactory(type, true); } return compFactories; }
public static ITypeTraits[] variablesToTypeTraits(Collection<LogicalVariable> varLogical, IVariableTypeEnvironment env, JobGenContext context) throws AlgebricksException { ITypeTraits[] typeTraits = new ITypeTraits[varLogical.size()]; ITypeTraitProvider typeTraitProvider = context.getTypeTraitProvider(); int i = 0; for (LogicalVariable v : varLogical) { Object type = env.getVarType(v); typeTraits[i++] = typeTraitProvider.getTypeTrait(type); } return typeTraits; }
public static IBinaryHashFunctionFactory[] variablesToBinaryHashFunctionFactories( Collection<LogicalVariable> varLogical, IVariableTypeEnvironment env, JobGenContext context) throws AlgebricksException { IBinaryHashFunctionFactory[] funFactories = new IBinaryHashFunctionFactory[varLogical.size()]; int i = 0; IBinaryHashFunctionFactoryProvider bhffProvider = context.getBinaryHashFunctionFactoryProvider(); for (LogicalVariable var : varLogical) { Object type = env.getVarType(var); funFactories[i++] = bhffProvider.getBinaryHashFunctionFactory(type); } return funFactories; }
public static IBinaryComparatorFactory[] variablesToAscBinaryComparatorFactories( Collection<LogicalVariable> varLogical, IVariableTypeEnvironment env, JobGenContext context) throws AlgebricksException { IBinaryComparatorFactory[] compFactories = new IBinaryComparatorFactory[varLogical.size()]; IBinaryComparatorFactoryProvider bcfProvider = context.getBinaryComparatorFactoryProvider(); int i = 0; for (LogicalVariable v : varLogical) { Object type = env.getVarType(v); compFactories[i++] = bcfProvider.getBinaryComparatorFactory(type, true); } return compFactories; }
public static ITypeTraits[] variablesToTypeTraits(List<LogicalVariable> varLogical, int start, int size, IVariableTypeEnvironment env, JobGenContext context) throws AlgebricksException { ITypeTraits[] typeTraits = new ITypeTraits[size]; ITypeTraitProvider typeTraitProvider = context.getTypeTraitProvider(); for (int i = 0; i < size; i++) { Object type = env.getVarType(varLogical.get(start + i)); typeTraits[i] = typeTraitProvider.getTypeTrait(type); } return typeTraits; }
public static INormalizedKeyComputerFactory variablesToAscNormalizedKeyComputerFactory( Collection<LogicalVariable> varLogical, IVariableTypeEnvironment env, JobGenContext context) throws AlgebricksException { INormalizedKeyComputerFactoryProvider nkcfProvider = context.getNormalizedKeyComputerFactoryProvider(); if (nkcfProvider == null) return null; for (LogicalVariable v : varLogical) { Object type = env.getVarType(v); return nkcfProvider.getNormalizedKeyComputerFactory(type, true); } return null; }
@Override public IVariableTypeEnvironment computeOutputTypeEnvironment(ITypingContext ctx) throws AlgebricksException { IVariableTypeEnvironment typeEnv = ctx.getOutputTypeEnvironment(inputs.get(0).getValue()); for (int i = 1; i < inputs.size(); i++) { checkTypeConsistency(typeEnv, compareVars.get(0), ctx.getOutputTypeEnvironment(inputs.get(i).getValue()), compareVars.get(i)); } IVariableTypeEnvironment env = new NonPropagatingTypeEnvironment(ctx.getExpressionTypeComputer(), ctx.getMetadataProvider()); int i = 0; for (; i < compareVars.get(0).size(); i++) { env.setVarType(outputVars.get(i), typeEnv.getVarType(compareVars.get(0).get(i))); } if (extraVars != null) { for (int k = 0; k < extraVars.get(0).size(); k++) { env.setVarType(outputVars.get(i + k), typeEnv.getVarType(extraVars.get(0).get(k))); } } return typeEnv; }
private Object getVarTypeFullList(LogicalVariable var, List<LogicalVariable> nonNullVariableList, List<List<LogicalVariable>> correlatedNullableVariableLists) throws AlgebricksException { Object t = varTypeMap.get(var); if (t != null) { return t; } for (Mutable<ILogicalOperator> r : op.getInputs()) { ILogicalOperator c = r.getValue(); IVariableTypeEnvironment env = ctx.getOutputTypeEnvironment(c); Object t2 = env.getVarType(var, nonNullVariableList, correlatedNullableVariableLists); if (t2 != null) { return t2; } } return null; }
public static IBinaryComparatorFactory[] variablesToBinaryComparatorFactories(Collection<OrderColumn> orderColumns, IVariableTypeEnvironment env, JobGenContext context) throws AlgebricksException { IBinaryComparatorFactory[] compFactories = new IBinaryComparatorFactory[orderColumns.size()]; IBinaryComparatorFactoryProvider bcfProvider = context.getBinaryComparatorFactoryProvider(); int i = 0; for (OrderColumn oc : orderColumns) { LogicalVariable v = oc.getColumn(); boolean ascending = oc.getOrder() == OrderOperator.IOrder.OrderKind.ASC; Object type = env.getVarType(v); compFactories[i++] = bcfProvider.getBinaryComparatorFactory(type, ascending); } return compFactories; }
private void setTypeTag(IOptimizationContext context, OptimizableOperatorSubTree subTree, IOptimizableFuncExpr optFuncExpr, int funcVarIndex) throws AlgebricksException { // Set the typeTag if the type is not null IAType type = (IAType) context.getOutputTypeEnvironment(subTree.getRoot()) .getVarType(optFuncExpr.getLogicalVar(funcVarIndex)); optFuncExpr.setFieldType(funcVarIndex, type); }
public static IPrinterFactory[] mkPrinterFactories(IOperatorSchema opSchema, IVariableTypeEnvironment env, JobGenContext context, int[] printColumns) throws AlgebricksException { IPrinterFactory[] pf = new IPrinterFactory[printColumns.length]; IPrinterFactoryProvider pff = context.getPrinterFactoryProvider(); try { for (int i = 0; i < pf.length; i++) { LogicalVariable v = opSchema.getVariable(printColumns[i]); Object t = env.getVarType(v); pf[i] = pff.getPrinterFactory(t); } return pf; } catch (HyracksDataException e) { throw new AlgebricksException(e); } }
@Override public Object getType(ILogicalExpression expr, IMetadataProvider<?, ?> metadataProvider, IVariableTypeEnvironment env) throws AlgebricksException { switch (expr.getExpressionTag()) { case CONSTANT: return getTypeForConstant((ConstantExpression) expr, env); case FUNCTION_CALL: return getTypeForFunction((AbstractFunctionCallExpression) expr, env, metadataProvider); case VARIABLE: try { return env.getVarType(((VariableReferenceExpression) expr).getVariableReference()); } catch (Exception e) { throw new CompilationException(ErrorCode.COMPILATION_ERROR, expr.getSourceLocation(), "Could not resolve type for " + expr.toString() + "," + "please check whether the used variable has been defined!", e); } default: throw new IllegalStateException(); } }
@Override public Pair<IConnectorDescriptor, TargetConstraint> createConnectorDescriptor(IConnectorDescriptorRegistry spec, ILogicalOperator op, IOperatorSchema opSchema, JobGenContext context) throws AlgebricksException { int[] keys = new int[hashFields.size()]; IBinaryHashFunctionFactory[] hashFunctionFactories = new IBinaryHashFunctionFactory[hashFields.size()]; int i = 0; IBinaryHashFunctionFactoryProvider hashFunProvider = context.getBinaryHashFunctionFactoryProvider(); IVariableTypeEnvironment env = context.getTypeEnvironment(op); for (LogicalVariable v : hashFields) { keys[i] = opSchema.findVariable(v); hashFunctionFactories[i] = hashFunProvider.getBinaryHashFunctionFactory(env.getVarType(v)); ++i; } ITuplePartitionComputerFactory tpcf = new FieldHashPartitionComputerFactory(keys, hashFunctionFactories); IConnectorDescriptor conn = new MToNPartitioningConnectorDescriptor(spec, tpcf); return new Pair<>(conn, null); }
@Override public ClosedDataInfo visitVariableReferenceExpression(VariableReferenceExpression expr, Void arg) throws AlgebricksException { Object varType = env.getVarType(expr.getVariableReference()); if (varType == null) { throw new CompilationException(ErrorCode.COMPILATION_ERROR, expr.getSourceLocation(), "Could not infer type for variable '" + expr.getVariableReference() + "'."); } return new ClosedDataInfo(false, TypeHelper.isClosed((IAType) varType), expr); }
@SuppressWarnings("rawtypes") public static RecordDescriptor mkRecordDescriptor(IVariableTypeEnvironment env, IOperatorSchema opSchema, JobGenContext context) throws AlgebricksException { ISerializerDeserializer[] fields = new ISerializerDeserializer[opSchema.getSize()]; ITypeTraits[] typeTraits = new ITypeTraits[opSchema.getSize()]; ISerializerDeserializerProvider sdp = context.getSerializerDeserializerProvider(); ITypeTraitProvider ttp = context.getTypeTraitProvider(); int i = 0; for (LogicalVariable var : opSchema) { Object t = env.getVarType(var); if (t == null) { LOGGER.warn("No type for variable " + var); } fields[i] = sdp.getSerializerDeserializer(t); typeTraits[i] = ttp.getTypeTrait(t); i++; } return new RecordDescriptor(fields, typeTraits); }
/** * Sets the subtree and the field type for the variables in the given function expression. * This method is only used for an arbitrary form of the probe-subtree in a join. */ protected void fillVarFieldTypeForOptFuncExprs(OptimizableOperatorSubTree subTree, AccessMethodAnalysisContext analysisCtx, ITypingContext context) throws AlgebricksException { ILogicalOperator rootOp = subTree.getRoot(); IVariableTypeEnvironment envSubtree = context.getOutputTypeEnvironment(rootOp); Set<LogicalVariable> liveVarsAtRootOp = new HashSet<LogicalVariable>(); VariableUtilities.getLiveVariables(rootOp, liveVarsAtRootOp); // For each optimizable function expression, applies the field type of each variable. for (IOptimizableFuncExpr optFuncExpr : analysisCtx.getMatchedFuncExprs()) { for (LogicalVariable var : liveVarsAtRootOp) { int optVarIndex = optFuncExpr.findLogicalVar(var); if (optVarIndex < 0) { continue; } optFuncExpr.setOptimizableSubTree(optVarIndex, subTree); IAType fieldType = (IAType) envSubtree.getVarType(var); optFuncExpr.setFieldType(optVarIndex, fieldType); } } }
@Override public Pair<IConnectorDescriptor, TargetConstraint> createConnectorDescriptor(IConnectorDescriptorRegistry spec, ILogicalOperator op, IOperatorSchema opSchema, JobGenContext context) throws AlgebricksException { int n = sortColumns.length; int[] sortFields = new int[n]; IBinaryComparatorFactory[] comps = new IBinaryComparatorFactory[n]; IBinaryHashFunctionFactory[] hashFuns = new IBinaryHashFunctionFactory[n]; IVariableTypeEnvironment env = context.getTypeEnvironment(op); INormalizedKeyComputerFactoryProvider nkcfProvider = context.getNormalizedKeyComputerFactoryProvider(); INormalizedKeyComputerFactory nkcf = null; for (int i = 0; i < n; i++) { sortFields[i] = opSchema.findVariable(sortColumns[i].getColumn()); Object type = env.getVarType(sortColumns[i].getColumn()); IBinaryComparatorFactoryProvider bcfp = context.getBinaryComparatorFactoryProvider(); comps[i] = bcfp.getBinaryComparatorFactory(type, sortColumns[i].getOrder() == OrderKind.ASC); IBinaryHashFunctionFactoryProvider bhffp = context.getBinaryHashFunctionFactoryProvider(); hashFuns[i] = bhffp.getBinaryHashFunctionFactory(type); if (i == 0 && nkcfProvider != null && type != null) { nkcf = nkcfProvider.getNormalizedKeyComputerFactory(type, sortColumns[i].getOrder() == OrderKind.ASC); } } ITuplePartitionComputerFactory tpcf = new FieldHashPartitionComputerFactory(sortFields, hashFuns); IConnectorDescriptor conn = new MToNPartitioningMergingConnectorDescriptor(spec, tpcf, sortFields, comps, nkcf); return new Pair<IConnectorDescriptor, TargetConstraint>(conn, TargetConstraint.ONE); }